[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:26:32 dist-test-slave-7f01 krb5kdc[5177](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:26:32 dist-test-slave-7f01 krb5kdc[5177](info): set up 2 sockets
Aug 08 22:26:32 dist-test-slave-7f01 krb5kdc[5177](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260808 22:26:33.368204 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.342s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:26:33 dist-test-slave-7f01 krb5kdc[5177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:26:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:26:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.11.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
I20260808 22:26:33.494764 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:40961
--webserver_interface=127.5.11.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:34617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:40961
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:26:33.605281 5201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:26:33.605564 5201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:26:33.605639 5201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:26:33.609524 5201 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:26:33.609614 5201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:26:33.609654 5201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:26:33.609723 5201 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:26:33.609763 5201 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:26:33.614723 5201 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:34617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:40961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:40961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:26:33.615680 5201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:26:33.616672 5201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:26:33.625394 5214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:33.625406 5211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:33.625470 5201 server_base.cc:1034] running on GCE node
W20260808 22:26:33.625411 5217 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:33.626287 5201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:26:33.627347 5201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:26:33.628547 5201 hybrid_clock.cc:648] HybridClock initialized: now 1786227993628447 us; error 113 us; skew 500 ppm
Aug 08 22:26:33 dist-test-slave-7f01 krb5kdc[5177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227993, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:26:33.631402 5201 init.cc:377] Logged in from keytab as kudu/127.5.11.254@KRBTEST.COM (short username kudu)
I20260808 22:26:33.632324 5201 webserver.cc:460] Webserver started at http://127.5.11.254:35637/ using document root <none> and password file <none>
I20260808 22:26:33.632838 5201 fs_manager.cc:359] Metadata directory not provided
I20260808 22:26:33.632895 5201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:26:33.633059 5201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:26:33.635003 5201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "ae70d804f17c4dcfbe240c7a39981524"
format_stamp: "Formatted at 2026-08-08 22:26:33 on dist-test-slave-7f01"
server_key: "a55ca01948133bb9e9cb7b590df3b368"
server_key_iv: "346fba8206b58bc9dcb5566d90a7440d"
server_key_version: "encryptionkey@0"
I20260808 22:26:33.635525 5201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "ae70d804f17c4dcfbe240c7a39981524"
format_stamp: "Formatted at 2026-08-08 22:26:33 on dist-test-slave-7f01"
server_key: "a55ca01948133bb9e9cb7b590df3b368"
server_key_iv: "346fba8206b58bc9dcb5566d90a7440d"
server_key_version: "encryptionkey@0"
I20260808 22:26:33.639097 5201 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 22:26:33.641446 5227 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:33.642644 5201 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:26:33.642777 5201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "ae70d804f17c4dcfbe240c7a39981524"
format_stamp: "Formatted at 2026-08-08 22:26:33 on dist-test-slave-7f01"
server_key: "a55ca01948133bb9e9cb7b590df3b368"
server_key_iv: "346fba8206b58bc9dcb5566d90a7440d"
server_key_version: "encryptionkey@0"
I20260808 22:26:33.642879 5201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:26:33.676904 5201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:26:33.680274 5201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:26:33.680521 5201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:26:33.688175 5201 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:40961
I20260808 22:26:33.688186 5312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:40961 every 8 connection(s)
I20260808 22:26:33.689157 5201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:26:33.691138 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 5201
I20260808 22:26:33.691325 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:26:33.692628 5313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:33.693076 5167 external_mini_cluster.cc:1428] Setting key 8f768a3362391193c3e1517327d99942
I20260808 22:26:33.699227 5313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae70d804f17c4dcfbe240c7a39981524: Bootstrap starting.
I20260808 22:26:33.701716 5313 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ae70d804f17c4dcfbe240c7a39981524: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:33.702648 5313 log.cc:824] T 00000000000000000000000000000000 P ae70d804f17c4dcfbe240c7a39981524: Log is configured to *not* fsync() on all Append() calls
I20260808 22:26:33.704648 5313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ae70d804f17c4dcfbe240c7a39981524: No bootstrap required, opened a new log
I20260808 22:26:33.709090 5313 raft_consensus.cc:348] T 00000000000000000000000000000000 P ae70d804f17c4dcfbe240c7a39981524 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae70d804f17c4dcfbe240c7a39981524" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 40961 } }
I20260808 22:26:33.709307 5313 raft_consensus.cc:374] T 00000000000000000000000000000000 P ae70d804f17c4dcfbe240c7a39981524 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:33.709393 5313 raft_consensus.cc:729] T 00000000000000000000000000000000 P ae70d804f17c4dcfbe240c7a39981524 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae70d804f17c4dcfbe240c7a39981524, State: Initialized, Role: FOLLOWER
I20260808 22:26:33.709846 5313 consensus_queue.cc:260] T 00000000000000000000000000000000 P ae70d804f17c4dcfbe240c7a39981524 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae70d804f17c4dcfbe240c7a39981524" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 40961 } }
I20260808 22:26:33.710016 5313 raft_consensus.cc:388] T 00000000000000000000000000000000 P ae70d804f17c4dcfbe240c7a39981524 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:26:33.710089 5313 raft_consensus.cc:482] T 00000000000000000000000000000000 P ae70d804f17c4dcfbe240c7a39981524 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:26:33.710197 5313 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ae70d804f17c4dcfbe240c7a39981524 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:33.711077 5313 raft_consensus.cc:504] T 00000000000000000000000000000000 P ae70d804f17c4dcfbe240c7a39981524 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae70d804f17c4dcfbe240c7a39981524" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 40961 } }
I20260808 22:26:33.711437 5313 leader_election.cc:302] T 00000000000000000000000000000000 P ae70d804f17c4dcfbe240c7a39981524 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ae70d804f17c4dcfbe240c7a39981524; no voters:
I20260808 22:26:33.711771 5313 leader_election.cc:288] T 00000000000000000000000000000000 P ae70d804f17c4dcfbe240c7a39981524 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:26:33.711894 5323 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ae70d804f17c4dcfbe240c7a39981524 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:26:33.712141 5323 raft_consensus.cc:686] T 00000000000000000000000000000000 P ae70d804f17c4dcfbe240c7a39981524 [term 1 LEADER]: Becoming Leader. State: Replica: ae70d804f17c4dcfbe240c7a39981524, State: Running, Role: LEADER
Aug 08 22:26:33 dist-test-slave-7f01 krb5kdc[5177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227993, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:26:33.712472 5323 consensus_queue.cc:237] T 00000000000000000000000000000000 P ae70d804f17c4dcfbe240c7a39981524 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae70d804f17c4dcfbe240c7a39981524" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 40961 } }
I20260808 22:26:33.713160 5313 sys_catalog.cc:564] T 00000000000000000000000000000000 P ae70d804f17c4dcfbe240c7a39981524 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:26:33.714210 5326 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae70d804f17c4dcfbe240c7a39981524 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ae70d804f17c4dcfbe240c7a39981524" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae70d804f17c4dcfbe240c7a39981524" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 40961 } } }
I20260808 22:26:33.714351 5326 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae70d804f17c4dcfbe240c7a39981524 [sys.catalog]: This master's current role is: LEADER
I20260808 22:26:33.714780 5327 sys_catalog.cc:455] T 00000000000000000000000000000000 P ae70d804f17c4dcfbe240c7a39981524 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ae70d804f17c4dcfbe240c7a39981524. Latest consensus state: current_term: 1 leader_uuid: "ae70d804f17c4dcfbe240c7a39981524" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ae70d804f17c4dcfbe240c7a39981524" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 40961 } } }
I20260808 22:26:33.715508 5327 sys_catalog.cc:458] T 00000000000000000000000000000000 P ae70d804f17c4dcfbe240c7a39981524 [sys.catalog]: This master's current role is: LEADER
W20260808 22:26:33.718480 5354 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ae70d804f17c4dcfbe240c7a39981524: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:26:33.718561 5354 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:26:33.718724 5339 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:26:33.719597 5339 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:26:33.719777 5317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:33.696896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34422 (local address 127.5.11.254:40961)
0808 22:26:33.697398 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:33.697422 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:33.698468 (+ 1046us) server_negotiation.cc:407] Connection header received
0808 22:26:33.701869 (+ 3401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:33.701893 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:33.702313 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:33.702671 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:33.705127 (+ 2456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:33.706656 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:33.707964 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:33.708679 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:33.708854 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:33.713306 (+ 4452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:33.713315 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:33.713332 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:33.713371 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:33.715792 (+ 2421us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:33.716396 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:33.716401 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:33.716405 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:33.716527 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:33.716915 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:33.716922 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:33.716924 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:33.717579 (+ 655us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:33.717793 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:33.718512 (+ 719us) server_negotiation.cc:299] Negotiation successful
0808 22:26:33.718808 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":151,"threads_started":1}
I20260808 22:26:33.725559 5339 catalog_manager.cc:1347] Generated new cluster ID: 657fba70c0214e319d2dcc0ea8080344
I20260808 22:26:33.725629 5339 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:26:33.758708 5339 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:26:33.760013 5339 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:26:33.768616 5339 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ae70d804f17c4dcfbe240c7a39981524: Generated new TSK 0
I20260808 22:26:33.769405 5339 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.11.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab.
Entry for principal kudu/127.5.11.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab.
I20260808 22:26:33.811168 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.193:0
--local_ip_for_outbound_sockets=127.5.11.193
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:40961
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:34617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:26:33.930308 5371 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:26:33.930562 5371 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:26:33.930604 5371 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:26:33.934857 5371 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:26:33.934921 5371 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:26:33.934978 5371 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.193
I20260808 22:26:33.940232 5371 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:34617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:40961
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5371
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.193
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:26:33.941270 5371 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:26:33.942293 5371 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:26:33.951809 5381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:33.951898 5371 server_base.cc:1034] running on GCE node
W20260808 22:26:33.951881 5380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:33.951809 5385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:33.952479 5371 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:26:33.953073 5371 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:26:33.954264 5371 hybrid_clock.cc:648] HybridClock initialized: now 1786227993954232 us; error 42 us; skew 500 ppm
Aug 08 22:26:33 dist-test-slave-7f01 krb5kdc[5177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227993, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:26:33.957134 5371 init.cc:377] Logged in from keytab as kudu/127.5.11.193@KRBTEST.COM (short username kudu)
I20260808 22:26:33.958091 5371 webserver.cc:460] Webserver started at http://127.5.11.193:45953/ using document root <none> and password file <none>
I20260808 22:26:33.958670 5371 fs_manager.cc:359] Metadata directory not provided
I20260808 22:26:33.958767 5371 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:26:33.959005 5371 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:26:33.961107 5371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-0/data/instance:
uuid: "eafef10f359c48e78e582aeb852ac4dc"
format_stamp: "Formatted at 2026-08-08 22:26:33 on dist-test-slave-7f01"
server_key: "f4c511626d82b2ffb7c2e663d5642e38"
server_key_iv: "252f52de66d4c969df3fa886f9323ab2"
server_key_version: "encryptionkey@0"
I20260808 22:26:33.961699 5371 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance:
uuid: "eafef10f359c48e78e582aeb852ac4dc"
format_stamp: "Formatted at 2026-08-08 22:26:33 on dist-test-slave-7f01"
server_key: "f4c511626d82b2ffb7c2e663d5642e38"
server_key_iv: "252f52de66d4c969df3fa886f9323ab2"
server_key_version: "encryptionkey@0"
I20260808 22:26:33.965502 5371 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260808 22:26:33.967911 5394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:33.969112 5371 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 22:26:33.969265 5371 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-0/wal
uuid: "eafef10f359c48e78e582aeb852ac4dc"
format_stamp: "Formatted at 2026-08-08 22:26:33 on dist-test-slave-7f01"
server_key: "f4c511626d82b2ffb7c2e663d5642e38"
server_key_iv: "252f52de66d4c969df3fa886f9323ab2"
server_key_version: "encryptionkey@0"
I20260808 22:26:33.969388 5371 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:26:33.978461 5371 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:26:33.981204 5371 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:26:33.981412 5371 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:26:33.981976 5371 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:26:33.982785 5371 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:26:33.982858 5371 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:33.982926 5371 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:26:33.982972 5371 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:33.992100 5371 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.193:43751
I20260808 22:26:33.992148 5602 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.193:43751 every 8 connection(s)
I20260808 22:26:33.992974 5371 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
I20260808 22:26:33.998083 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 5371
I20260808 22:26:33.998204 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance
I20260808 22:26:33.998447 5167 external_mini_cluster.cc:1428] Setting key deef3b4847a898d59de8cc49ff4e0412
Aug 08 22:26:33 dist-test-slave-7f01 krb5kdc[5177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227993, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:26:34.008101 5317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:33.994814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:58897 (local address 127.5.11.254:40961)
0808 22:26:33.994952 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:33.994956 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:33.995713 (+ 757us) server_negotiation.cc:407] Connection header received
0808 22:26:33.997094 (+ 1381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:33.997097 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:33.997162 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:33.997269 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:33.998506 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:33.999174 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:34.000719 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:34.000953 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:34.001043 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:34.004150 (+ 3107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:34.004153 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:34.004155 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:34.004179 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:34.005713 (+ 1534us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:34.006364 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:34.006366 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:34.006367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:34.006446 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:34.006896 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:34.006899 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:34.006900 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:34.007068 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:34.007140 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:34.007855 (+ 715us) server_negotiation.cc:299] Negotiation successful
0808 22:26:34.007981 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 22:26:34.009204 5608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:33.995104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40961 (local address 127.5.11.193:58897)
0808 22:26:33.995561 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:33.995605 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:33.996876 (+ 1271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:33.997451 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:33.997458 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:33.997808 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:33.998357 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:33.998372 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:33.999324 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:33.999327 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:34.000583 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:34.000598 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:34.001119 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:34.001126 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:34.001369 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:34.001992 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:26:34.002018 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:34.003972 (+ 1954us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:34.005837 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:34.005844 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:34.005861 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:34.006258 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:34.006543 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:34.006548 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:34.006554 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:34.006763 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:34.007178 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:34.007187 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:34.007527 (+ 340us) client_negotiation.cc:769] Sending connection context
0808 22:26:34.007776 (+ 249us) client_negotiation.cc:240] Negotiation successful
0808 22:26:34.008144 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":109,"threads_started":1}
WARNING: no policy specified for kudu/127.5.11.194@KRBTEST.COM; defaulting to no policy
I20260808 22:26:34.010485 5603 heartbeater.cc:347] Connected to a master server at 127.5.11.254:40961
I20260808 22:26:34.010738 5603 heartbeater.cc:464] Registering TS with master...
I20260808 22:26:34.011265 5603 heartbeater.cc:511] Master 127.5.11.254:40961 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.194@KRBTEST.COM" created.
I20260808 22:26:34.012876 5256 ts_manager.cc:194] Registered new tserver with Master: eafef10f359c48e78e582aeb852ac4dc (127.5.11.193:43751)
I20260808 22:26:34.014505 5256 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.193@KRBTEST.COM'} at 127.5.11.193:58897
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab.
Entry for principal kudu/127.5.11.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab.
I20260808 22:26:34.028906 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.194:0
--local_ip_for_outbound_sockets=127.5.11.194
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:40961
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:34617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:26:34.140568 5613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:26:34.140836 5613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:26:34.140909 5613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:26:34.144682 5613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:26:34.144761 5613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:26:34.144845 5613 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.194
I20260808 22:26:34.150009 5613 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:34617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:40961
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.194
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:26:34.150930 5613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:26:34.151918 5613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:26:34.161158 5624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:34.161223 5623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:34.161228 5629 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:34.161440 5613 server_base.cc:1034] running on GCE node
I20260808 22:26:34.161877 5613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:26:34.162490 5613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:26:34.163683 5613 hybrid_clock.cc:648] HybridClock initialized: now 1786227994163669 us; error 25 us; skew 500 ppm
Aug 08 22:26:34 dist-test-slave-7f01 krb5kdc[5177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227994, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:26:34.172408 5613 init.cc:377] Logged in from keytab as kudu/127.5.11.194@KRBTEST.COM (short username kudu)
I20260808 22:26:34.173297 5613 webserver.cc:460] Webserver started at http://127.5.11.194:44061/ using document root <none> and password file <none>
I20260808 22:26:34.173882 5613 fs_manager.cc:359] Metadata directory not provided
I20260808 22:26:34.174002 5613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:26:34.174228 5613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:26:34.176235 5613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-1/data/instance:
uuid: "284c85e617164fccb54768abfc66966c"
format_stamp: "Formatted at 2026-08-08 22:26:34 on dist-test-slave-7f01"
server_key: "e76136f3e06f8d7c02650b9ddad85576"
server_key_iv: "e2d7b102f5a87cd2f938d7fb225e6075"
server_key_version: "encryptionkey@0"
I20260808 22:26:34.176795 5613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance:
uuid: "284c85e617164fccb54768abfc66966c"
format_stamp: "Formatted at 2026-08-08 22:26:34 on dist-test-slave-7f01"
server_key: "e76136f3e06f8d7c02650b9ddad85576"
server_key_iv: "e2d7b102f5a87cd2f938d7fb225e6075"
server_key_version: "encryptionkey@0"
I20260808 22:26:34.180431 5613 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 22:26:34.182761 5644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:34.184017 5613 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260808 22:26:34.184137 5613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-1/wal
uuid: "284c85e617164fccb54768abfc66966c"
format_stamp: "Formatted at 2026-08-08 22:26:34 on dist-test-slave-7f01"
server_key: "e76136f3e06f8d7c02650b9ddad85576"
server_key_iv: "e2d7b102f5a87cd2f938d7fb225e6075"
server_key_version: "encryptionkey@0"
I20260808 22:26:34.184242 5613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:26:34.197281 5613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:26:34.199954 5613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:26:34.200145 5613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:26:34.200685 5613 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:26:34.201464 5613 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:26:34.201516 5613 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:34.201581 5613 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:26:34.201618 5613 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:34.210870 5613 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.194:36971
I20260808 22:26:34.210932 5840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.194:36971 every 8 connection(s)
I20260808 22:26:34.211716 5613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
I20260808 22:26:34.215161 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 5613
I20260808 22:26:34.215258 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance
I20260808 22:26:34.215605 5167 external_mini_cluster.cc:1428] Setting key cd4b1cd9ca45a756284f21b7f0f27f5c
Aug 08 22:26:34 dist-test-slave-7f01 krb5kdc[5177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227994, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.11.195@KRBTEST.COM; defaulting to no policy
I20260808 22:26:34.227391 5317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:34.213700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:57217 (local address 127.5.11.254:40961)
0808 22:26:34.213850 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:34.213856 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:34.214815 (+ 959us) server_negotiation.cc:407] Connection header received
0808 22:26:34.216618 (+ 1803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:34.216620 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:34.216664 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:34.216728 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:34.218033 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:34.218870 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:34.220394 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:34.220801 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:34.220930 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:34.223833 (+ 2903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:34.223836 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:34.223838 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:34.223862 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:34.225311 (+ 1449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:34.225840 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:34.225848 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:34.225849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:34.225925 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:34.226301 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:34.226305 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:34.226306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:34.226485 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:34.226591 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:34.227099 (+ 508us) server_negotiation.cc:299] Negotiation successful
0808 22:26:34.227219 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 22:26:34.228370 5844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:34.214101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40961 (local address 127.5.11.194:57217)
0808 22:26:34.214669 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:34.214710 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:34.216390 (+ 1680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:34.216938 (+ 548us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:34.216948 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:34.217408 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:34.217873 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:34.217886 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:34.218997 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:34.219000 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:34.220252 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:34.220268 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:34.220971 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:34.220978 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:34.221203 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:34.221821 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:26:34.221847 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:34.223640 (+ 1793us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:34.225436 (+ 1796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:34.225442 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:34.225455 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:34.225726 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:34.226036 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:34.226039 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:34.226041 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:34.226194 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:34.226590 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:34.226597 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:34.226848 (+ 251us) client_negotiation.cc:769] Sending connection context
0808 22:26:34.227037 (+ 189us) client_negotiation.cc:240] Negotiation successful
0808 22:26:34.227347 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":109,"threads_started":1}
I20260808 22:26:34.229564 5842 heartbeater.cc:347] Connected to a master server at 127.5.11.254:40961
I20260808 22:26:34.229818 5842 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.195@KRBTEST.COM" created.
I20260808 22:26:34.230381 5842 heartbeater.cc:511] Master 127.5.11.254:40961 requested a full tablet report, sending...
I20260808 22:26:34.231523 5256 ts_manager.cc:194] Registered new tserver with Master: 284c85e617164fccb54768abfc66966c (127.5.11.194:36971)
I20260808 22:26:34.232182 5256 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.194@KRBTEST.COM'} at 127.5.11.194:57217
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab.
Entry for principal kudu/127.5.11.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab.
I20260808 22:26:34.246109 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.195:0
--local_ip_for_outbound_sockets=127.5.11.195
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:40961
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:34617
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:26:34.358472 5849 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:26:34.358695 5849 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:26:34.358736 5849 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:26:34.362756 5849 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:26:34.362811 5849 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:26:34.362871 5849 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.195
I20260808 22:26:34.367918 5849 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:34617
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:40961
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5849
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.195
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:26:34.368841 5849 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:26:34.369791 5849 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:26:34.379333 5864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:34.379506 5849 server_base.cc:1034] running on GCE node
W20260808 22:26:34.379359 5863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:34.379685 5868 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:34.380180 5849 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:26:34.380764 5849 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:26:34.381948 5849 hybrid_clock.cc:648] HybridClock initialized: now 1786227994381935 us; error 34 us; skew 500 ppm
Aug 08 22:26:34 dist-test-slave-7f01 krb5kdc[5177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227994, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:26:34.384789 5849 init.cc:377] Logged in from keytab as kudu/127.5.11.195@KRBTEST.COM (short username kudu)
I20260808 22:26:34.385605 5849 webserver.cc:460] Webserver started at http://127.5.11.195:38695/ using document root <none> and password file <none>
I20260808 22:26:34.386155 5849 fs_manager.cc:359] Metadata directory not provided
I20260808 22:26:34.386247 5849 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:26:34.386483 5849 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:26:34.388619 5849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-2/data/instance:
uuid: "dc7fd9a18e2f465195d5392ef12f2549"
format_stamp: "Formatted at 2026-08-08 22:26:34 on dist-test-slave-7f01"
server_key: "e13c8449620a3c8b36f99c98f9ce0893"
server_key_iv: "2366e4494c02b138b0e1784a00cd12f3"
server_key_version: "encryptionkey@0"
I20260808 22:26:34.389215 5849 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance:
uuid: "dc7fd9a18e2f465195d5392ef12f2549"
format_stamp: "Formatted at 2026-08-08 22:26:34 on dist-test-slave-7f01"
server_key: "e13c8449620a3c8b36f99c98f9ce0893"
server_key_iv: "2366e4494c02b138b0e1784a00cd12f3"
server_key_version: "encryptionkey@0"
I20260808 22:26:34.392854 5849 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260808 22:26:34.395258 5877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:34.396441 5849 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:26:34.396572 5849 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-2/wal
uuid: "dc7fd9a18e2f465195d5392ef12f2549"
format_stamp: "Formatted at 2026-08-08 22:26:34 on dist-test-slave-7f01"
server_key: "e13c8449620a3c8b36f99c98f9ce0893"
server_key_iv: "2366e4494c02b138b0e1784a00cd12f3"
server_key_version: "encryptionkey@0"
I20260808 22:26:34.396677 5849 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:26:34.416277 5849 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:26:34.419426 5849 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:26:34.419613 5849 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:26:34.420210 5849 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:26:34.421008 5849 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:26:34.421082 5849 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:34.421150 5849 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:26:34.421198 5849 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:34.429936 5849 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.195:43861
I20260808 22:26:34.429963 6086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.195:43861 every 8 connection(s)
I20260808 22:26:34.430760 5849 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
I20260808 22:26:34.432152 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 5849
I20260808 22:26:34.432308 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance
I20260808 22:26:34.432639 5167 external_mini_cluster.cc:1428] Setting key cb16ae63482016a11cd3b6b2d3e422b9
Aug 08 22:26:34 dist-test-slave-7f01 krb5kdc[5177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227994, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:26:34.444056 5317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:34.432852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:54551 (local address 127.5.11.254:40961)
0808 22:26:34.433007 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:34.433012 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:34.433970 (+ 958us) server_negotiation.cc:407] Connection header received
0808 22:26:34.435344 (+ 1374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:34.435347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:34.435404 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:34.435481 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:34.436635 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:34.437153 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:34.438285 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:34.438526 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:34.438636 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:34.441050 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:34.441053 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:34.441055 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:34.441079 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:34.442297 (+ 1218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:34.442757 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:34.442759 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:34.442760 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:34.442835 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:34.443122 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:34.443124 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:34.443125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:34.443287 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:34.443362 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:34.443818 (+ 456us) server_negotiation.cc:299] Negotiation successful
0808 22:26:34.443936 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 22:26:34.444832 6091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:34.433339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40961 (local address 127.5.11.195:54551)
0808 22:26:34.433821 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:34.433884 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:34.435138 (+ 1254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:34.435644 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:34.435651 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:34.436049 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:34.436490 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:34.436503 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:34.437285 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:34.437288 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:34.438158 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:34.438168 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:34.438675 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:34.438679 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:34.438847 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:34.439450 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:26:34.439466 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:34.440899 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:34.442401 (+ 1502us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:34.442406 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:34.442416 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:34.442670 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:34.442915 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:34.442917 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:34.442918 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:34.443044 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:34.443375 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:34.443380 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:34.443581 (+ 201us) client_negotiation.cc:769] Sending connection context
0808 22:26:34.443724 (+ 143us) client_negotiation.cc:240] Negotiation successful
0808 22:26:34.444047 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":133,"threads_started":1}
I20260808 22:26:34.445951 6087 heartbeater.cc:347] Connected to a master server at 127.5.11.254:40961
I20260808 22:26:34.446183 6087 heartbeater.cc:464] Registering TS with master...
I20260808 22:26:34.446782 6087 heartbeater.cc:511] Master 127.5.11.254:40961 requested a full tablet report, sending...
I20260808 22:26:34.447827 5256 ts_manager.cc:194] Registered new tserver with Master: dc7fd9a18e2f465195d5392ef12f2549 (127.5.11.195:43861)
I20260808 22:26:34.448333 5256 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.195@KRBTEST.COM'} at 127.5.11.195:54551
I20260808 22:26:34.458393 5167 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 22:26:34 dist-test-slave-7f01 krb5kdc[5177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227994, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 22:26:34 dist-test-slave-7f01 krb5kdc[5177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227994, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.11.193@KRBTEST.COM
I20260808 22:26:34.484953 6102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:34.475977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33434 (local address 127.5.11.193:43751)
0808 22:26:34.476282 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:34.476295 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:34.476318 (+ 23us) server_negotiation.cc:407] Connection header received
0808 22:26:34.476436 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:34.476442 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:34.476611 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:34.476725 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:34.477334 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:34.478090 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:34.478871 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:34.479213 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:34.479293 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:34.481617 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:34.481626 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:34.481637 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:34.481659 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:34.483300 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:34.483672 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:34.483677 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:34.483678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:34.483808 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:34.484048 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:34.484050 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:34.484051 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:34.484284 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:34.484456 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:34.484591 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 22:26:34.484754 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":101,"threads_started":1}
W20260808 22:26:34.485924 5441 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33434
Aug 08 22:26:34 dist-test-slave-7f01 krb5kdc[5177](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 22:26:34 dist-test-slave-7f01 krb5kdc[5177](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227994, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.193@KRBTEST.COM
I20260808 22:26:34.511474 6102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:34.503854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33448 (local address 127.5.11.193:43751)
0808 22:26:34.503990 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:34.503996 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:34.504136 (+ 140us) server_negotiation.cc:407] Connection header received
0808 22:26:34.504227 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:34.504230 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:34.504268 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:34.504334 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:34.504975 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:34.505465 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:34.506214 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:34.506424 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:34.506494 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:34.508670 (+ 2176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:34.508673 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:34.508675 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:34.508696 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:34.510058 (+ 1362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:34.510428 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:34.510431 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:34.510432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:34.510503 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:34.510726 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:34.510728 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:34.510729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:34.510876 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:34.510971 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:34.511171 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 22:26:34.511259 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 22:26:34.512657 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 5371
I20260808 22:26:34.522233 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 5613
I20260808 22:26:34.537621 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 5849
I20260808 22:26:34.550014 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 5201
2026-08-08T22:26:34Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2558 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:26:34 dist-test-slave-7f01 krb5kdc[6116](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:26:34 dist-test-slave-7f01 krb5kdc[6116](info): set up 2 sockets
Aug 08 22:26:34 dist-test-slave-7f01 krb5kdc[6116](info): commencing operation
krb5kdc: starting...
W20260808 22:26:35.799330 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:26:35 dist-test-slave-7f01 krb5kdc[6116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:26:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:26:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.11.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
I20260808 22:26:35.927047 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:44753
--webserver_interface=127.5.11.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:41727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:44753
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:26:36.039166 6136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:26:36.039391 6136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:26:36.039428 6136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:26:36.043253 6136 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:26:36.043301 6136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:26:36.043320 6136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:26:36.043336 6136 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:26:36.043351 6136 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:26:36.048272 6136 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:41727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:44753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:44753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:26:36.049100 6136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:26:36.049976 6136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:26:36.057968 6145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:36.057930 6149 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:36.058189 6136 server_base.cc:1034] running on GCE node
W20260808 22:26:36.057968 6144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:36.058784 6136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:26:36.059809 6136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:26:36.061017 6136 hybrid_clock.cc:648] HybridClock initialized: now 1786227996060995 us; error 43 us; skew 500 ppm
Aug 08 22:26:36 dist-test-slave-7f01 krb5kdc[6116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227996, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:26:36.063911 6136 init.cc:377] Logged in from keytab as kudu/127.5.11.254@KRBTEST.COM (short username kudu)
I20260808 22:26:36.064817 6136 webserver.cc:460] Webserver started at http://127.5.11.254:34319/ using document root <none> and password file <none>
I20260808 22:26:36.065393 6136 fs_manager.cc:359] Metadata directory not provided
I20260808 22:26:36.065485 6136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:26:36.065713 6136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:26:36.067711 6136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "95f6c0414da94ca9ad81acfb342a6f51"
format_stamp: "Formatted at 2026-08-08 22:26:36 on dist-test-slave-7f01"
server_key: "a98afe140d9b37d0e8d19527a91e2e17"
server_key_iv: "ce192e874397f2019948643c2e31413e"
server_key_version: "encryptionkey@0"
I20260808 22:26:36.068357 6136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "95f6c0414da94ca9ad81acfb342a6f51"
format_stamp: "Formatted at 2026-08-08 22:26:36 on dist-test-slave-7f01"
server_key: "a98afe140d9b37d0e8d19527a91e2e17"
server_key_iv: "ce192e874397f2019948643c2e31413e"
server_key_version: "encryptionkey@0"
I20260808 22:26:36.072155 6136 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 22:26:36.074549 6157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:36.075810 6136 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:26:36.075963 6136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "95f6c0414da94ca9ad81acfb342a6f51"
format_stamp: "Formatted at 2026-08-08 22:26:36 on dist-test-slave-7f01"
server_key: "a98afe140d9b37d0e8d19527a91e2e17"
server_key_iv: "ce192e874397f2019948643c2e31413e"
server_key_version: "encryptionkey@0"
I20260808 22:26:36.076073 6136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:26:36.087767 6136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:26:36.090593 6136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:26:36.090787 6136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:26:36.098189 6136 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:44753
I20260808 22:26:36.098217 6239 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:44753 every 8 connection(s)
I20260808 22:26:36.099092 6136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:26:36.101778 6240 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:36.103024 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 6136
I20260808 22:26:36.103116 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:26:36.103407 5167 external_mini_cluster.cc:1428] Setting key 83a0d43e27b11dfac2fbbf0d8334043d
I20260808 22:26:36.108467 6240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95f6c0414da94ca9ad81acfb342a6f51: Bootstrap starting.
I20260808 22:26:36.110935 6240 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95f6c0414da94ca9ad81acfb342a6f51: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:36.111652 6240 log.cc:824] T 00000000000000000000000000000000 P 95f6c0414da94ca9ad81acfb342a6f51: Log is configured to *not* fsync() on all Append() calls
Aug 08 22:26:36 dist-test-slave-7f01 krb5kdc[6116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:26:36.113790 6240 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95f6c0414da94ca9ad81acfb342a6f51: No bootstrap required, opened a new log
I20260808 22:26:36.118543 6244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:36.104820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45470 (local address 127.5.11.254:44753)
0808 22:26:36.105253 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:36.105272 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:36.105314 (+ 42us) server_negotiation.cc:407] Connection header received
0808 22:26:36.106140 (+ 826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:36.106165 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:36.106577 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:36.106951 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:36.107578 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:36.109001 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:36.109833 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:36.110491 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:36.110648 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:36.113420 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:36.113426 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:36.113441 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:36.113472 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:36.115442 (+ 1970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:36.115936 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:36.115943 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:36.115948 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:36.116109 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:36.116379 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:36.116384 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:36.116387 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:36.116896 (+ 509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:36.117094 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:36.117287 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 22:26:36.117571 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":116,"threads_started":1}
I20260808 22:26:36.118523 6240 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95f6c0414da94ca9ad81acfb342a6f51 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95f6c0414da94ca9ad81acfb342a6f51" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 44753 } }
I20260808 22:26:36.118806 6240 raft_consensus.cc:374] T 00000000000000000000000000000000 P 95f6c0414da94ca9ad81acfb342a6f51 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:36.118868 6240 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95f6c0414da94ca9ad81acfb342a6f51 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95f6c0414da94ca9ad81acfb342a6f51, State: Initialized, Role: FOLLOWER
I20260808 22:26:36.119326 6240 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95f6c0414da94ca9ad81acfb342a6f51 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95f6c0414da94ca9ad81acfb342a6f51" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 44753 } }
I20260808 22:26:36.119484 6240 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95f6c0414da94ca9ad81acfb342a6f51 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:26:36.119555 6240 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95f6c0414da94ca9ad81acfb342a6f51 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:26:36.119656 6240 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95f6c0414da94ca9ad81acfb342a6f51 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:36.120573 6240 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95f6c0414da94ca9ad81acfb342a6f51 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95f6c0414da94ca9ad81acfb342a6f51" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 44753 } }
I20260808 22:26:36.120929 6240 leader_election.cc:302] T 00000000000000000000000000000000 P 95f6c0414da94ca9ad81acfb342a6f51 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95f6c0414da94ca9ad81acfb342a6f51; no voters:
I20260808 22:26:36.121184 6240 leader_election.cc:288] T 00000000000000000000000000000000 P 95f6c0414da94ca9ad81acfb342a6f51 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:26:36.121330 6246 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95f6c0414da94ca9ad81acfb342a6f51 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:26:36.121611 6246 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95f6c0414da94ca9ad81acfb342a6f51 [term 1 LEADER]: Becoming Leader. State: Replica: 95f6c0414da94ca9ad81acfb342a6f51, State: Running, Role: LEADER
I20260808 22:26:36.121918 6246 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95f6c0414da94ca9ad81acfb342a6f51 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95f6c0414da94ca9ad81acfb342a6f51" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 44753 } }
I20260808 22:26:36.122434 6240 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95f6c0414da94ca9ad81acfb342a6f51 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:26:36.123628 6249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95f6c0414da94ca9ad81acfb342a6f51 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95f6c0414da94ca9ad81acfb342a6f51. Latest consensus state: current_term: 1 leader_uuid: "95f6c0414da94ca9ad81acfb342a6f51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95f6c0414da94ca9ad81acfb342a6f51" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 44753 } } }
I20260808 22:26:36.123597 6247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95f6c0414da94ca9ad81acfb342a6f51 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "95f6c0414da94ca9ad81acfb342a6f51" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95f6c0414da94ca9ad81acfb342a6f51" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 44753 } } }
I20260808 22:26:36.123765 6249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95f6c0414da94ca9ad81acfb342a6f51 [sys.catalog]: This master's current role is: LEADER
I20260808 22:26:36.123777 6247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95f6c0414da94ca9ad81acfb342a6f51 [sys.catalog]: This master's current role is: LEADER
I20260808 22:26:36.124236 6258 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:26:36.127327 6258 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:26:36.132858 6258 catalog_manager.cc:1347] Generated new cluster ID: acc5d75559564c6abe602094c5dff140
I20260808 22:26:36.132932 6258 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:26:36.152092 6258 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:26:36.152908 6258 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:26:36.164755 6258 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 95f6c0414da94ca9ad81acfb342a6f51: Generated new TSK 0
I20260808 22:26:36.165401 6258 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.11.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab.
Entry for principal kudu/127.5.11.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab.
I20260808 22:26:36.200681 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.193:0
--local_ip_for_outbound_sockets=127.5.11.193
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:44753
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:41727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:26:36.311884 6292 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:26:36.312124 6292 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:26:36.312165 6292 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:26:36.315968 6292 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:26:36.316012 6292 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:26:36.316067 6292 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.193
I20260808 22:26:36.320966 6292 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:41727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:44753
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6292
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.193
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:26:36.321839 6292 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:26:36.322801 6292 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:26:36.332119 6298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:36.332106 6299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:36.332337 6301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:36.332399 6292 server_base.cc:1034] running on GCE node
I20260808 22:26:36.332777 6292 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:26:36.333389 6292 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:26:36.334583 6292 hybrid_clock.cc:648] HybridClock initialized: now 1786227996334570 us; error 27 us; skew 500 ppm
Aug 08 22:26:36 dist-test-slave-7f01 krb5kdc[6116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227996, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:26:36.337424 6292 init.cc:377] Logged in from keytab as kudu/127.5.11.193@KRBTEST.COM (short username kudu)
I20260808 22:26:36.338217 6292 webserver.cc:460] Webserver started at http://127.5.11.193:38069/ using document root <none> and password file <none>
I20260808 22:26:36.338706 6292 fs_manager.cc:359] Metadata directory not provided
I20260808 22:26:36.338761 6292 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:26:36.338919 6292 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:26:36.340796 6292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-0/data/instance:
uuid: "49530c3a11df466997dcc95c6224817c"
format_stamp: "Formatted at 2026-08-08 22:26:36 on dist-test-slave-7f01"
server_key: "a82426182182c471952c321c1318f32a"
server_key_iv: "3b50ec879e9d3e934cfe856da07d8a03"
server_key_version: "encryptionkey@0"
I20260808 22:26:36.341277 6292 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance:
uuid: "49530c3a11df466997dcc95c6224817c"
format_stamp: "Formatted at 2026-08-08 22:26:36 on dist-test-slave-7f01"
server_key: "a82426182182c471952c321c1318f32a"
server_key_iv: "3b50ec879e9d3e934cfe856da07d8a03"
server_key_version: "encryptionkey@0"
I20260808 22:26:36.344915 6292 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:26:36.347205 6313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:36.348438 6292 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:26:36.348562 6292 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-0/wal
uuid: "49530c3a11df466997dcc95c6224817c"
format_stamp: "Formatted at 2026-08-08 22:26:36 on dist-test-slave-7f01"
server_key: "a82426182182c471952c321c1318f32a"
server_key_iv: "3b50ec879e9d3e934cfe856da07d8a03"
server_key_version: "encryptionkey@0"
I20260808 22:26:36.348661 6292 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:26:36.358505 6292 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:26:36.361195 6292 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:26:36.361402 6292 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:26:36.361954 6292 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:26:36.362746 6292 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:26:36.362795 6292 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:36.362865 6292 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:26:36.362898 6292 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:36.371779 6292 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.193:39857
I20260808 22:26:36.371821 6512 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.193:39857 every 8 connection(s)
I20260808 22:26:36.372640 6292 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
I20260808 22:26:36.376895 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 6292
I20260808 22:26:36.377027 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance
I20260808 22:26:36.377278 5167 external_mini_cluster.cc:1428] Setting key 820e0c320ba8ee5bbf0618363932d900
Aug 08 22:26:36 dist-test-slave-7f01 krb5kdc[6116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227996, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:26:36.387290 6244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:36.374341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:60347 (local address 127.5.11.254:44753)
0808 22:26:36.374479 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:36.374483 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:36.375187 (+ 704us) server_negotiation.cc:407] Connection header received
0808 22:26:36.376596 (+ 1409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:36.376600 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:36.376664 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:36.376765 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:36.378087 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:36.378698 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:36.380061 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:36.380469 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:36.380596 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:36.383244 (+ 2648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:36.383251 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:36.383257 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:36.383292 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:36.385092 (+ 1800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:36.385625 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:36.385632 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:36.385636 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:36.385742 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:36.386083 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:36.386088 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:36.386093 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:36.386345 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:36.386469 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:36.386937 (+ 468us) server_negotiation.cc:299] Negotiation successful
0808 22:26:36.387075 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 22:26:36.388005 6519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:36.374620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:44753 (local address 127.5.11.193:60347)
0808 22:26:36.375038 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:36.375080 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:36.376375 (+ 1295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:36.376938 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:36.376945 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:36.377412 (+ 467us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:36.377932 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:36.377949 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:36.378834 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:36.378837 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:36.379882 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:36.379894 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:36.380612 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:36.380615 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:36.380808 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:36.381414 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:26:36.381439 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:36.383080 (+ 1641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:36.385238 (+ 2158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:36.385243 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:36.385253 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:36.385518 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:36.385850 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:36.385852 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:36.385854 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:36.385985 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:36.386464 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:36.386469 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:36.386679 (+ 210us) client_negotiation.cc:769] Sending connection context
0808 22:26:36.386850 (+ 171us) client_negotiation.cc:240] Negotiation successful
0808 22:26:36.387090 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":102,"threads_started":1}
WARNING: no policy specified for kudu/127.5.11.194@KRBTEST.COM; defaulting to no policy
I20260808 22:26:36.389215 6513 heartbeater.cc:347] Connected to a master server at 127.5.11.254:44753
I20260808 22:26:36.389467 6513 heartbeater.cc:464] Registering TS with master...
I20260808 22:26:36.390046 6513 heartbeater.cc:511] Master 127.5.11.254:44753 requested a full tablet report, sending...
I20260808 22:26:36.391666 6188 ts_manager.cc:194] Registered new tserver with Master: 49530c3a11df466997dcc95c6224817c (127.5.11.193:39857)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.194@KRBTEST.COM" created.
I20260808 22:26:36.392920 6188 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.193@KRBTEST.COM'} at 127.5.11.193:60347
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab.
Entry for principal kudu/127.5.11.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab.
I20260808 22:26:36.407155 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.194:0
--local_ip_for_outbound_sockets=127.5.11.194
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:44753
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:41727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:26:36.517688 6523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:26:36.517971 6523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:26:36.518052 6523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:26:36.522010 6523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:26:36.522091 6523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:26:36.522207 6523 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.194
I20260808 22:26:36.527676 6523 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:41727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:44753
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.194
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:26:36.528765 6523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:26:36.529758 6523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:26:36.539005 6531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:36.539026 6532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:36.539013 6534 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:36.539234 6523 server_base.cc:1034] running on GCE node
I20260808 22:26:36.539702 6523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:26:36.540383 6523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:26:36.541554 6523 hybrid_clock.cc:648] HybridClock initialized: now 1786227996541541 us; error 25 us; skew 500 ppm
Aug 08 22:26:36 dist-test-slave-7f01 krb5kdc[6116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227996, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:26:36.544778 6523 init.cc:377] Logged in from keytab as kudu/127.5.11.194@KRBTEST.COM (short username kudu)
I20260808 22:26:36.545625 6523 webserver.cc:460] Webserver started at http://127.5.11.194:46003/ using document root <none> and password file <none>
I20260808 22:26:36.546155 6523 fs_manager.cc:359] Metadata directory not provided
I20260808 22:26:36.546214 6523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:26:36.546375 6523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:26:36.548413 6523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-1/data/instance:
uuid: "2aeda88e2ab0477ba816f641cf47d07d"
format_stamp: "Formatted at 2026-08-08 22:26:36 on dist-test-slave-7f01"
server_key: "cc94871bb5bf5fada7160500f7735967"
server_key_iv: "796caa95a165a727388db5cc149c5d8b"
server_key_version: "encryptionkey@0"
I20260808 22:26:36.548962 6523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance:
uuid: "2aeda88e2ab0477ba816f641cf47d07d"
format_stamp: "Formatted at 2026-08-08 22:26:36 on dist-test-slave-7f01"
server_key: "cc94871bb5bf5fada7160500f7735967"
server_key_iv: "796caa95a165a727388db5cc149c5d8b"
server_key_version: "encryptionkey@0"
I20260808 22:26:36.552543 6523 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 22:26:36.554883 6550 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:36.556150 6523 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:26:36.556277 6523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-1/wal
uuid: "2aeda88e2ab0477ba816f641cf47d07d"
format_stamp: "Formatted at 2026-08-08 22:26:36 on dist-test-slave-7f01"
server_key: "cc94871bb5bf5fada7160500f7735967"
server_key_iv: "796caa95a165a727388db5cc149c5d8b"
server_key_version: "encryptionkey@0"
I20260808 22:26:36.556381 6523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:26:36.576107 6523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:26:36.578833 6523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:26:36.579056 6523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:26:36.579650 6523 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:26:36.580507 6523 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:26:36.580583 6523 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:36.580653 6523 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:26:36.580708 6523 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:36.589612 6523 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.194:44707
I20260808 22:26:36.589637 6757 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.194:44707 every 8 connection(s)
I20260808 22:26:36.590490 6523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
I20260808 22:26:36.593276 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 6523
I20260808 22:26:36.593406 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance
I20260808 22:26:36.593715 5167 external_mini_cluster.cc:1428] Setting key e6bead319f9575878d3c2f2add59734d
Aug 08 22:26:36 dist-test-slave-7f01 krb5kdc[6116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227996, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.11.195@KRBTEST.COM; defaulting to no policy
I20260808 22:26:36.605954 6244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:36.592325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:40225 (local address 127.5.11.254:44753)
0808 22:26:36.592463 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:36.592467 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:36.593354 (+ 887us) server_negotiation.cc:407] Connection header received
0808 22:26:36.595002 (+ 1648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:36.595006 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:36.595062 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:36.595144 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:36.596573 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:36.597092 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:36.598825 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:36.599046 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:36.599146 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:36.602202 (+ 3056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:36.602205 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:36.602207 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:36.602231 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:36.603705 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:36.604292 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:36.604295 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:36.604297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:36.604386 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:36.604756 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:36.604760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:36.604762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:36.605013 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:36.605125 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:36.605617 (+ 492us) server_negotiation.cc:299] Negotiation successful
0808 22:26:36.605780 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 22:26:36.606703 6762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:36.592634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:44753 (local address 127.5.11.194:40225)
0808 22:26:36.593161 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:36.593223 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:36.594789 (+ 1566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:36.595315 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:36.595322 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:36.595674 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:36.596406 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:36.596421 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:36.597230 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:36.597238 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:36.598667 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:36.598682 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:36.599202 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:36.599209 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:36.599447 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:36.600225 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:26:36.600251 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:36.602016 (+ 1765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:36.603884 (+ 1868us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:36.603890 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:36.603905 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:36.604178 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:36.604481 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:36.604484 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:36.604486 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:36.604653 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:36.605122 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:36.605129 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:36.605371 (+ 242us) client_negotiation.cc:769] Sending connection context
0808 22:26:36.605572 (+ 201us) client_negotiation.cc:240] Negotiation successful
0808 22:26:36.605890 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":150,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.195@KRBTEST.COM" created.
I20260808 22:26:36.608012 6759 heartbeater.cc:347] Connected to a master server at 127.5.11.254:44753
I20260808 22:26:36.608258 6759 heartbeater.cc:464] Registering TS with master...
I20260808 22:26:36.608737 6759 heartbeater.cc:511] Master 127.5.11.254:44753 requested a full tablet report, sending...
I20260808 22:26:36.609858 6188 ts_manager.cc:194] Registered new tserver with Master: 2aeda88e2ab0477ba816f641cf47d07d (127.5.11.194:44707)
I20260808 22:26:36.610431 6188 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.194@KRBTEST.COM'} at 127.5.11.194:40225
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab.
Entry for principal kudu/127.5.11.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab.
I20260808 22:26:36.622848 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.195:0
--local_ip_for_outbound_sockets=127.5.11.195
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:44753
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:41727
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:26:36.736495 6766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:26:36.736724 6766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:26:36.736766 6766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:26:36.740768 6766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:26:36.740826 6766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:26:36.740882 6766 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.195
I20260808 22:26:36.745941 6766 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:41727
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:44753
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.195
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:26:36.746856 6766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:26:36.747840 6766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:26:36.757266 6779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:36.757272 6774 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:36.757290 6776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:36.757615 6766 server_base.cc:1034] running on GCE node
I20260808 22:26:36.758105 6766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:26:36.758782 6766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:26:36.759953 6766 hybrid_clock.cc:648] HybridClock initialized: now 1786227996759923 us; error 39 us; skew 500 ppm
Aug 08 22:26:36 dist-test-slave-7f01 krb5kdc[6116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227996, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:26:36.762744 6766 init.cc:377] Logged in from keytab as kudu/127.5.11.195@KRBTEST.COM (short username kudu)
I20260808 22:26:36.763628 6766 webserver.cc:460] Webserver started at http://127.5.11.195:42835/ using document root <none> and password file <none>
I20260808 22:26:36.764205 6766 fs_manager.cc:359] Metadata directory not provided
I20260808 22:26:36.764268 6766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:26:36.764515 6766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:26:36.766489 6766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-2/data/instance:
uuid: "f28025aa32014d7cac111e5c83e0c1dd"
format_stamp: "Formatted at 2026-08-08 22:26:36 on dist-test-slave-7f01"
server_key: "e4b6c8e2688316c61d98ea4b793ee0fa"
server_key_iv: "754085d61ce084963b3f416fbbb38932"
server_key_version: "encryptionkey@0"
I20260808 22:26:36.767061 6766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance:
uuid: "f28025aa32014d7cac111e5c83e0c1dd"
format_stamp: "Formatted at 2026-08-08 22:26:36 on dist-test-slave-7f01"
server_key: "e4b6c8e2688316c61d98ea4b793ee0fa"
server_key_iv: "754085d61ce084963b3f416fbbb38932"
server_key_version: "encryptionkey@0"
I20260808 22:26:36.772761 6766 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260808 22:26:36.775202 6790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:36.776413 6766 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:26:36.776546 6766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-2/wal
uuid: "f28025aa32014d7cac111e5c83e0c1dd"
format_stamp: "Formatted at 2026-08-08 22:26:36 on dist-test-slave-7f01"
server_key: "e4b6c8e2688316c61d98ea4b793ee0fa"
server_key_iv: "754085d61ce084963b3f416fbbb38932"
server_key_version: "encryptionkey@0"
I20260808 22:26:36.776665 6766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:26:36.807698 6766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:26:36.811234 6766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:26:36.811443 6766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:26:36.812091 6766 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:26:36.812940 6766 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:26:36.813023 6766 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:36.813119 6766 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:26:36.813171 6766 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:36.822170 6766 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.195:46235
I20260808 22:26:36.822230 6991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.195:46235 every 8 connection(s)
I20260808 22:26:36.823221 6766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
I20260808 22:26:36.829556 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 6766
I20260808 22:26:36.829679 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance
I20260808 22:26:36.829964 5167 external_mini_cluster.cc:1428] Setting key ce9ce2c842a93cec37b2c0615314cad0
Aug 08 22:26:36 dist-test-slave-7f01 krb5kdc[6116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227996, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:26:36.837105 6244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:36.825258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:53105 (local address 127.5.11.254:44753)
0808 22:26:36.825402 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:36.825406 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:36.826229 (+ 823us) server_negotiation.cc:407] Connection header received
0808 22:26:36.827626 (+ 1397us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:36.827630 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:36.827692 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:36.827836 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:36.828998 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:36.829681 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:36.831058 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:36.831313 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:36.831398 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:36.833892 (+ 2494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:36.833895 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:36.833897 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:36.833920 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:36.835170 (+ 1250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:36.835640 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:36.835643 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:36.835644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:36.835722 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:36.836079 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:36.836082 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:36.836083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:36.836256 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:36.836332 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:36.836885 (+ 553us) server_negotiation.cc:299] Negotiation successful
0808 22:26:36.836979 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 22:26:36.837886 6995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:36.825634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:44753 (local address 127.5.11.195:53105)
0808 22:26:36.826082 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:36.826124 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:36.827400 (+ 1276us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:36.828019 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:36.828027 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:36.828372 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:36.828848 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:36.828862 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:36.829806 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:36.829809 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:36.830944 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:36.830954 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:36.831480 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:36.831484 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:36.831625 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:36.832291 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:26:36.832307 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:36.833742 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:36.835274 (+ 1532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:36.835279 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:36.835290 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:36.835554 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:36.835856 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:36.835858 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:36.835860 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:36.835995 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:36.836345 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:36.836350 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:36.836658 (+ 308us) client_negotiation.cc:769] Sending connection context
0808 22:26:36.836835 (+ 177us) client_negotiation.cc:240] Negotiation successful
0808 22:26:36.837079 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":113,"threads_started":1}
I20260808 22:26:36.839015 6992 heartbeater.cc:347] Connected to a master server at 127.5.11.254:44753
I20260808 22:26:36.839258 6992 heartbeater.cc:464] Registering TS with master...
I20260808 22:26:36.839797 6992 heartbeater.cc:511] Master 127.5.11.254:44753 requested a full tablet report, sending...
I20260808 22:26:36.840811 6188 ts_manager.cc:194] Registered new tserver with Master: f28025aa32014d7cac111e5c83e0c1dd (127.5.11.195:46235)
I20260808 22:26:36.841323 6188 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.195@KRBTEST.COM'} at 127.5.11.195:53105
I20260808 22:26:36.844115 5167 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:26:36.854709 6244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:36.847552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45484 (local address 127.5.11.254:44753)
0808 22:26:36.847699 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:36.847704 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:36.847962 (+ 258us) server_negotiation.cc:407] Connection header received
0808 22:26:36.848050 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:36.848052 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:36.848090 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:36.848154 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:36.848767 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:36.849245 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:36.850026 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:36.850223 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:36.850294 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:36.851533 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:36.851536 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:36.851538 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:36.851560 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:36.853072 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:36.853531 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:36.853534 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:36.853535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:36.853617 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:36.853867 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:36.853869 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:36.853870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:36.854076 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:36.854179 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:36.854392 (+ 213us) server_negotiation.cc:299] Negotiation successful
0808 22:26:36.854517 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 22:26:36.859845 6188 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45484:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:26:36.863086 6188 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:26:36.877784 7012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:36.871414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:44707 (local address 127.0.0.1:57296)
0808 22:26:36.871928 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:36.871977 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:36.872112 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:36.873180 (+ 1068us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:36.873188 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:36.873211 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:26:36.873365 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:36.873374 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:36.874794 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:36.874797 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:36.876319 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:36.876331 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:36.877254 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:36.877258 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:36.877362 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:36.877400 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 22:26:36.877515 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 22:26:36.877629 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":96,"thread_start_us":69,"threads_started":1}
I20260808 22:26:36.877825 7028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:36.872128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:39857 (local address 127.0.0.1:55056)
0808 22:26:36.872896 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:36.872909 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:36.872996 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:36.873693 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:36.873699 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:36.873719 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:26:36.873803 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:36.873808 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:36.875154 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:36.875158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:36.876294 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:36.876304 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:36.877493 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:36.877497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:36.877579 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:36.877599 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 22:26:36.877663 (+ 64us) client_negotiation.cc:240] Negotiation successful
0808 22:26:36.877720 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"mutex_wait_us":3,"thread_start_us":86,"threads_started":1}
I20260808 22:26:36.877830 7025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:36.871560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57296 (local address 127.5.11.194:44707)
0808 22:26:36.872537 (+ 977us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:36.872549 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:36.872573 (+ 24us) server_negotiation.cc:407] Connection header received
0808 22:26:36.872656 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:36.872661 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:36.873007 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:36.873171 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:26:36.873490 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:36.874664 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:36.876467 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:36.877116 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:36.877226 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:36.877363 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:36.877563 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 22:26:36.877658 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":867,"thread_start_us":135,"threads_started":1}
I20260808 22:26:36.878216 7031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:36.872523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55056 (local address 127.5.11.193:39857)
0808 22:26:36.873231 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:36.873245 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:36.873266 (+ 21us) server_negotiation.cc:407] Connection header received
0808 22:26:36.873356 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:36.873360 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:36.873549 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:36.873698 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:26:36.873962 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:36.875000 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:36.876424 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:36.877149 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:36.877276 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:36.877426 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:36.877943 (+ 517us) server_negotiation.cc:299] Negotiation successful
0808 22:26:36.878064 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":104,"threads_started":1}
I20260808 22:26:36.878434 7014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:36.871606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:46235 (local address 127.0.0.1:48596)
0808 22:26:36.872023 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:36.872039 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:36.872144 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:36.873507 (+ 1363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:36.873509 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:36.873524 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:26:36.873596 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:36.873600 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:36.874678 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:36.874684 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:36.876777 (+ 2093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:36.876787 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:36.878014 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:36.878021 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:36.878122 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:36.878146 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 22:26:36.878220 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 22:26:36.878302 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"mutex_wait_us":182,"thread_start_us":88,"threads_started":1}
I20260808 22:26:36.878970 7030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:36.872371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48596 (local address 127.5.11.195:46235)
0808 22:26:36.872901 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:36.872910 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:36.872929 (+ 19us) server_negotiation.cc:407] Connection header received
0808 22:26:36.872991 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:36.872994 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:36.873170 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:36.873326 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:26:36.873740 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:36.874537 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:36.876908 (+ 2371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:36.877701 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:36.877805 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:36.877945 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:36.878723 (+ 778us) server_negotiation.cc:299] Negotiation successful
0808 22:26:36.878843 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":184,"threads_started":1}
I20260808 22:26:36.881537 6879 tablet_service.cc:1467] Processing CreateTablet for tablet 25f5c3ebfb9b471aa73afbede0cb6fee (DEFAULT_TABLE table=test-table [id=f35bf3968f824f5a90f60af145215a56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:26:36.881557 6394 tablet_service.cc:1467] Processing CreateTablet for tablet 25f5c3ebfb9b471aa73afbede0cb6fee (DEFAULT_TABLE table=test-table [id=f35bf3968f824f5a90f60af145215a56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:26:36.881556 6633 tablet_service.cc:1467] Processing CreateTablet for tablet 25f5c3ebfb9b471aa73afbede0cb6fee (DEFAULT_TABLE table=test-table [id=f35bf3968f824f5a90f60af145215a56]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:26:36.882565 6879 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25f5c3ebfb9b471aa73afbede0cb6fee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:36.882581 6633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25f5c3ebfb9b471aa73afbede0cb6fee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:36.882581 6394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 25f5c3ebfb9b471aa73afbede0cb6fee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:36.889411 7035 tablet_bootstrap.cc:492] T 25f5c3ebfb9b471aa73afbede0cb6fee P f28025aa32014d7cac111e5c83e0c1dd: Bootstrap starting.
I20260808 22:26:36.889878 7037 tablet_bootstrap.cc:492] T 25f5c3ebfb9b471aa73afbede0cb6fee P 2aeda88e2ab0477ba816f641cf47d07d: Bootstrap starting.
I20260808 22:26:36.891832 7035 tablet_bootstrap.cc:654] T 25f5c3ebfb9b471aa73afbede0cb6fee P f28025aa32014d7cac111e5c83e0c1dd: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:36.892068 7036 tablet_bootstrap.cc:492] T 25f5c3ebfb9b471aa73afbede0cb6fee P 49530c3a11df466997dcc95c6224817c: Bootstrap starting.
I20260808 22:26:36.892505 7037 tablet_bootstrap.cc:654] T 25f5c3ebfb9b471aa73afbede0cb6fee P 2aeda88e2ab0477ba816f641cf47d07d: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:36.893260 7037 log.cc:824] T 25f5c3ebfb9b471aa73afbede0cb6fee P 2aeda88e2ab0477ba816f641cf47d07d: Log is configured to *not* fsync() on all Append() calls
I20260808 22:26:36.893260 7035 log.cc:824] T 25f5c3ebfb9b471aa73afbede0cb6fee P f28025aa32014d7cac111e5c83e0c1dd: Log is configured to *not* fsync() on all Append() calls
I20260808 22:26:36.893919 7036 tablet_bootstrap.cc:654] T 25f5c3ebfb9b471aa73afbede0cb6fee P 49530c3a11df466997dcc95c6224817c: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:36.894832 7036 log.cc:824] T 25f5c3ebfb9b471aa73afbede0cb6fee P 49530c3a11df466997dcc95c6224817c: Log is configured to *not* fsync() on all Append() calls
I20260808 22:26:36.895432 7035 tablet_bootstrap.cc:492] T 25f5c3ebfb9b471aa73afbede0cb6fee P f28025aa32014d7cac111e5c83e0c1dd: No bootstrap required, opened a new log
I20260808 22:26:36.895432 7037 tablet_bootstrap.cc:492] T 25f5c3ebfb9b471aa73afbede0cb6fee P 2aeda88e2ab0477ba816f641cf47d07d: No bootstrap required, opened a new log
I20260808 22:26:36.895643 7035 ts_tablet_manager.cc:1397] T 25f5c3ebfb9b471aa73afbede0cb6fee P f28025aa32014d7cac111e5c83e0c1dd: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.006s
I20260808 22:26:36.895643 7037 ts_tablet_manager.cc:1397] T 25f5c3ebfb9b471aa73afbede0cb6fee P 2aeda88e2ab0477ba816f641cf47d07d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 22:26:36.896530 7036 tablet_bootstrap.cc:492] T 25f5c3ebfb9b471aa73afbede0cb6fee P 49530c3a11df466997dcc95c6224817c: No bootstrap required, opened a new log
I20260808 22:26:36.896733 7036 ts_tablet_manager.cc:1397] T 25f5c3ebfb9b471aa73afbede0cb6fee P 49530c3a11df466997dcc95c6224817c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 22:26:36.899562 7035 raft_consensus.cc:348] T 25f5c3ebfb9b471aa73afbede0cb6fee P f28025aa32014d7cac111e5c83e0c1dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aeda88e2ab0477ba816f641cf47d07d" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 44707 } } peers { permanent_uuid: "f28025aa32014d7cac111e5c83e0c1dd" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 46235 } } peers { permanent_uuid: "49530c3a11df466997dcc95c6224817c" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39857 } }
I20260808 22:26:36.899562 7037 raft_consensus.cc:348] T 25f5c3ebfb9b471aa73afbede0cb6fee P 2aeda88e2ab0477ba816f641cf47d07d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aeda88e2ab0477ba816f641cf47d07d" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 44707 } } peers { permanent_uuid: "f28025aa32014d7cac111e5c83e0c1dd" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 46235 } } peers { permanent_uuid: "49530c3a11df466997dcc95c6224817c" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39857 } }
I20260808 22:26:36.899868 7037 raft_consensus.cc:374] T 25f5c3ebfb9b471aa73afbede0cb6fee P 2aeda88e2ab0477ba816f641cf47d07d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:36.899868 7035 raft_consensus.cc:374] T 25f5c3ebfb9b471aa73afbede0cb6fee P f28025aa32014d7cac111e5c83e0c1dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:36.899696 7036 raft_consensus.cc:348] T 25f5c3ebfb9b471aa73afbede0cb6fee P 49530c3a11df466997dcc95c6224817c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aeda88e2ab0477ba816f641cf47d07d" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 44707 } } peers { permanent_uuid: "f28025aa32014d7cac111e5c83e0c1dd" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 46235 } } peers { permanent_uuid: "49530c3a11df466997dcc95c6224817c" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39857 } }
I20260808 22:26:36.899932 7035 raft_consensus.cc:729] T 25f5c3ebfb9b471aa73afbede0cb6fee P f28025aa32014d7cac111e5c83e0c1dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f28025aa32014d7cac111e5c83e0c1dd, State: Initialized, Role: FOLLOWER
I20260808 22:26:36.899961 7037 raft_consensus.cc:729] T 25f5c3ebfb9b471aa73afbede0cb6fee P 2aeda88e2ab0477ba816f641cf47d07d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2aeda88e2ab0477ba816f641cf47d07d, State: Initialized, Role: FOLLOWER
I20260808 22:26:36.900048 7036 raft_consensus.cc:374] T 25f5c3ebfb9b471aa73afbede0cb6fee P 49530c3a11df466997dcc95c6224817c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:36.900139 7036 raft_consensus.cc:729] T 25f5c3ebfb9b471aa73afbede0cb6fee P 49530c3a11df466997dcc95c6224817c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49530c3a11df466997dcc95c6224817c, State: Initialized, Role: FOLLOWER
I20260808 22:26:36.900416 7035 consensus_queue.cc:260] T 25f5c3ebfb9b471aa73afbede0cb6fee P f28025aa32014d7cac111e5c83e0c1dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aeda88e2ab0477ba816f641cf47d07d" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 44707 } } peers { permanent_uuid: "f28025aa32014d7cac111e5c83e0c1dd" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 46235 } } peers { permanent_uuid: "49530c3a11df466997dcc95c6224817c" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39857 } }
I20260808 22:26:36.900416 7037 consensus_queue.cc:260] T 25f5c3ebfb9b471aa73afbede0cb6fee P 2aeda88e2ab0477ba816f641cf47d07d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aeda88e2ab0477ba816f641cf47d07d" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 44707 } } peers { permanent_uuid: "f28025aa32014d7cac111e5c83e0c1dd" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 46235 } } peers { permanent_uuid: "49530c3a11df466997dcc95c6224817c" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39857 } }
I20260808 22:26:36.900664 7036 consensus_queue.cc:260] T 25f5c3ebfb9b471aa73afbede0cb6fee P 49530c3a11df466997dcc95c6224817c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aeda88e2ab0477ba816f641cf47d07d" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 44707 } } peers { permanent_uuid: "f28025aa32014d7cac111e5c83e0c1dd" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 46235 } } peers { permanent_uuid: "49530c3a11df466997dcc95c6224817c" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39857 } }
I20260808 22:26:36.901412 7037 ts_tablet_manager.cc:1428] T 25f5c3ebfb9b471aa73afbede0cb6fee P 2aeda88e2ab0477ba816f641cf47d07d: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260808 22:26:36.901412 7035 ts_tablet_manager.cc:1428] T 25f5c3ebfb9b471aa73afbede0cb6fee P f28025aa32014d7cac111e5c83e0c1dd: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 22:26:36.901685 7036 ts_tablet_manager.cc:1428] T 25f5c3ebfb9b471aa73afbede0cb6fee P 49530c3a11df466997dcc95c6224817c: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 22:26:36.901801 6759 heartbeater.cc:503] Master 127.5.11.254:44753 was elected leader, sending a full tablet report...
I20260808 22:26:36.902043 6513 heartbeater.cc:503] Master 127.5.11.254:44753 was elected leader, sending a full tablet report...
I20260808 22:26:36.902868 6992 heartbeater.cc:503] Master 127.5.11.254:44753 was elected leader, sending a full tablet report...
I20260808 22:26:36.992928 7044 raft_consensus.cc:482] T 25f5c3ebfb9b471aa73afbede0cb6fee P 2aeda88e2ab0477ba816f641cf47d07d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:26:36.993188 7044 raft_consensus.cc:504] T 25f5c3ebfb9b471aa73afbede0cb6fee P 2aeda88e2ab0477ba816f641cf47d07d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aeda88e2ab0477ba816f641cf47d07d" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 44707 } } peers { permanent_uuid: "f28025aa32014d7cac111e5c83e0c1dd" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 46235 } } peers { permanent_uuid: "49530c3a11df466997dcc95c6224817c" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39857 } }
I20260808 22:26:36.994481 7044 leader_election.cc:288] T 25f5c3ebfb9b471aa73afbede0cb6fee P 2aeda88e2ab0477ba816f641cf47d07d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f28025aa32014d7cac111e5c83e0c1dd (127.5.11.195:46235), 49530c3a11df466997dcc95c6224817c (127.5.11.193:39857)
I20260808 22:26:36.999279 7030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:36.994825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:57343 (local address 127.5.11.195:46235)
0808 22:26:36.994987 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:36.994992 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:36.995067 (+ 75us) server_negotiation.cc:407] Connection header received
0808 22:26:36.995219 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:36.995221 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:36.995265 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:36.995357 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:26:36.995668 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:36.996446 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:36.998055 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:36.998655 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:36.998774 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:36.998828 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:36.999071 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 22:26:36.999142 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 22:26:36.999279 7031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:36.994912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:45441 (local address 127.5.11.193:39857)
0808 22:26:36.995036 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:36.995043 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:36.995360 (+ 317us) server_negotiation.cc:407] Connection header received
0808 22:26:36.995498 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:36.995502 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:36.995546 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:36.995647 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:26:36.995953 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:36.996669 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:36.998058 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:36.998650 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:36.998775 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:36.998828 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:36.999071 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 22:26:36.999142 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 22:26:36.999297 6762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:36.994690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:46235 (local address 127.5.11.194:57343)
0808 22:26:36.994994 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:36.995010 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:36.995123 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:36.995390 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:36.995393 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:36.995410 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:26:36.995548 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:36.995553 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:36.996572 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:36.996575 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:36.997887 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:36.997899 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:36.998779 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:36.998783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:36.998923 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:36.998960 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 22:26:36.999035 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 22:26:36.999142 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"mutex_wait_us":66}
I20260808 22:26:36.999306 7048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:36.994774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:39857 (local address 127.5.11.194:45441)
0808 22:26:36.995256 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:36.995268 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:36.995384 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:36.995663 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:36.995667 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:36.995686 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:26:36.995834 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:36.995840 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:36.996772 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:36.996776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:36.997950 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:36.997960 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:36.998779 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:36.998783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:36.998919 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:36.998954 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 22:26:36.999035 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 22:26:36.999144 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"mutex_wait_us":81,"thread_start_us":156,"threads_started":1}
I20260808 22:26:37.000063 6920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25f5c3ebfb9b471aa73afbede0cb6fee" candidate_uuid: "2aeda88e2ab0477ba816f641cf47d07d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f28025aa32014d7cac111e5c83e0c1dd" is_pre_election: true
I20260808 22:26:37.000289 6430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25f5c3ebfb9b471aa73afbede0cb6fee" candidate_uuid: "2aeda88e2ab0477ba816f641cf47d07d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49530c3a11df466997dcc95c6224817c" is_pre_election: true
I20260808 22:26:37.000394 6920 raft_consensus.cc:2454] T 25f5c3ebfb9b471aa73afbede0cb6fee P f28025aa32014d7cac111e5c83e0c1dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2aeda88e2ab0477ba816f641cf47d07d in term 0.
I20260808 22:26:37.000599 6430 raft_consensus.cc:2454] T 25f5c3ebfb9b471aa73afbede0cb6fee P 49530c3a11df466997dcc95c6224817c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2aeda88e2ab0477ba816f641cf47d07d in term 0.
I20260808 22:26:37.000939 6554 leader_election.cc:302] T 25f5c3ebfb9b471aa73afbede0cb6fee P 2aeda88e2ab0477ba816f641cf47d07d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2aeda88e2ab0477ba816f641cf47d07d, f28025aa32014d7cac111e5c83e0c1dd; no voters:
I20260808 22:26:37.001233 7044 raft_consensus.cc:2781] T 25f5c3ebfb9b471aa73afbede0cb6fee P 2aeda88e2ab0477ba816f641cf47d07d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:26:37.001325 7044 raft_consensus.cc:482] T 25f5c3ebfb9b471aa73afbede0cb6fee P 2aeda88e2ab0477ba816f641cf47d07d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:26:37.001370 7044 raft_consensus.cc:3037] T 25f5c3ebfb9b471aa73afbede0cb6fee P 2aeda88e2ab0477ba816f641cf47d07d [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:37.002409 7044 raft_consensus.cc:504] T 25f5c3ebfb9b471aa73afbede0cb6fee P 2aeda88e2ab0477ba816f641cf47d07d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aeda88e2ab0477ba816f641cf47d07d" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 44707 } } peers { permanent_uuid: "f28025aa32014d7cac111e5c83e0c1dd" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 46235 } } peers { permanent_uuid: "49530c3a11df466997dcc95c6224817c" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39857 } }
I20260808 22:26:37.002753 7044 leader_election.cc:288] T 25f5c3ebfb9b471aa73afbede0cb6fee P 2aeda88e2ab0477ba816f641cf47d07d [CANDIDATE]: Term 1 election: Requested vote from peers f28025aa32014d7cac111e5c83e0c1dd (127.5.11.195:46235), 49530c3a11df466997dcc95c6224817c (127.5.11.193:39857)
I20260808 22:26:37.003060 6920 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25f5c3ebfb9b471aa73afbede0cb6fee" candidate_uuid: "2aeda88e2ab0477ba816f641cf47d07d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f28025aa32014d7cac111e5c83e0c1dd"
I20260808 22:26:37.003123 6430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "25f5c3ebfb9b471aa73afbede0cb6fee" candidate_uuid: "2aeda88e2ab0477ba816f641cf47d07d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49530c3a11df466997dcc95c6224817c"
I20260808 22:26:37.003227 6920 raft_consensus.cc:3037] T 25f5c3ebfb9b471aa73afbede0cb6fee P f28025aa32014d7cac111e5c83e0c1dd [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:37.003237 6430 raft_consensus.cc:3037] T 25f5c3ebfb9b471aa73afbede0cb6fee P 49530c3a11df466997dcc95c6224817c [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:37.004359 6430 raft_consensus.cc:2454] T 25f5c3ebfb9b471aa73afbede0cb6fee P 49530c3a11df466997dcc95c6224817c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2aeda88e2ab0477ba816f641cf47d07d in term 1.
I20260808 22:26:37.004391 6920 raft_consensus.cc:2454] T 25f5c3ebfb9b471aa73afbede0cb6fee P f28025aa32014d7cac111e5c83e0c1dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2aeda88e2ab0477ba816f641cf47d07d in term 1.
I20260808 22:26:37.004678 6551 leader_election.cc:302] T 25f5c3ebfb9b471aa73afbede0cb6fee P 2aeda88e2ab0477ba816f641cf47d07d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2aeda88e2ab0477ba816f641cf47d07d, 49530c3a11df466997dcc95c6224817c; no voters:
I20260808 22:26:37.004854 7044 raft_consensus.cc:2781] T 25f5c3ebfb9b471aa73afbede0cb6fee P 2aeda88e2ab0477ba816f641cf47d07d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:26:37.005113 7044 raft_consensus.cc:686] T 25f5c3ebfb9b471aa73afbede0cb6fee P 2aeda88e2ab0477ba816f641cf47d07d [term 1 LEADER]: Becoming Leader. State: Replica: 2aeda88e2ab0477ba816f641cf47d07d, State: Running, Role: LEADER
I20260808 22:26:37.005445 7044 consensus_queue.cc:237] T 25f5c3ebfb9b471aa73afbede0cb6fee P 2aeda88e2ab0477ba816f641cf47d07d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aeda88e2ab0477ba816f641cf47d07d" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 44707 } } peers { permanent_uuid: "f28025aa32014d7cac111e5c83e0c1dd" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 46235 } } peers { permanent_uuid: "49530c3a11df466997dcc95c6224817c" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39857 } }
I20260808 22:26:37.008805 6188 catalog_manager.cc:5495] T 25f5c3ebfb9b471aa73afbede0cb6fee P 2aeda88e2ab0477ba816f641cf47d07d reported cstate change: term changed from 0 to 1, leader changed from <none> to 2aeda88e2ab0477ba816f641cf47d07d (127.5.11.194). New cstate: current_term: 1 leader_uuid: "2aeda88e2ab0477ba816f641cf47d07d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2aeda88e2ab0477ba816f641cf47d07d" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 44707 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f28025aa32014d7cac111e5c83e0c1dd" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 46235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49530c3a11df466997dcc95c6224817c" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39857 } health_report { overall_health: UNKNOWN } } }
Aug 08 22:26:37 dist-test-slave-7f01 krb5kdc[6116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227995, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.193@KRBTEST.COM
I20260808 22:26:37.031075 7031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:37.022473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55058 (local address 127.5.11.193:39857)
0808 22:26:37.022668 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:37.022673 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:37.022861 (+ 188us) server_negotiation.cc:407] Connection header received
0808 22:26:37.022960 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:37.022962 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:37.023003 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:37.023069 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:37.023902 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:37.024404 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:37.025234 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:37.025433 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:37.025517 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:37.027985 (+ 2468us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:37.027997 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:37.028011 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:37.028032 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:37.029515 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:37.029906 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:37.029910 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:37.029912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:37.029990 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:37.030221 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:37.030223 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:37.030224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:37.030453 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:37.030613 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:37.030750 (+ 137us) server_negotiation.cc:299] Negotiation successful
0808 22:26:37.030839 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
Aug 08 22:26:37 dist-test-slave-7f01 krb5kdc[6116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227997, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 22:26:37 dist-test-slave-7f01 krb5kdc[6116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227997, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.11.193@KRBTEST.COM
I20260808 22:26:37.058677 7031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:37.050690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55066 (local address 127.5.11.193:39857)
0808 22:26:37.050833 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:37.050838 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:37.051014 (+ 176us) server_negotiation.cc:407] Connection header received
0808 22:26:37.051175 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:37.051178 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:37.051215 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:37.051296 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:37.052070 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:37.052652 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:37.053409 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:37.053635 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:37.053714 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:37.055978 (+ 2264us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:37.055981 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:37.055983 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:37.056004 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:37.057214 (+ 1210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:37.057583 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:37.057585 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:37.057586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:37.057661 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:37.057895 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:37.057897 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:37.057899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:37.058045 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:37.058144 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:37.058368 (+ 224us) server_negotiation.cc:299] Negotiation successful
0808 22:26:37.058495 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
Aug 08 22:26:37 dist-test-slave-7f01 krb5kdc[6116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260808 22:26:37.074216 6994 tablet.cc:2369] T 25f5c3ebfb9b471aa73afbede0cb6fee P f28025aa32014d7cac111e5c83e0c1dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 22:26:37 dist-test-slave-7f01 krb5kdc[6116](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.193@KRBTEST.COM
I20260808 22:26:37.083783 7031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:37.076001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55074 (local address 127.5.11.193:39857)
0808 22:26:37.076175 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:37.076179 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:37.076254 (+ 75us) server_negotiation.cc:407] Connection header received
0808 22:26:37.076368 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:37.076370 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:37.076407 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:37.076473 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:37.077138 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:37.077598 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:37.078373 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:37.078568 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:37.078639 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:37.081105 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:37.081108 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:37.081110 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:37.081130 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:37.082377 (+ 1247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:37.082746 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:37.082749 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:37.082750 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:37.082823 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:37.083056 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:37.083058 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:37.083059 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:37.083202 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:37.083299 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:37.083463 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 22:26:37.083553 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
W20260808 22:26:37.085094 6365 tablet_service.cc:2908] Rejecting scan request for tablet 25f5c3ebfb9b471aa73afbede0cb6fee: Uninitialized: clean time has not yet been initialized
I20260808 22:26:37.087055 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 6292
W20260808 22:26:37.092748 6760 tablet.cc:2369] T 25f5c3ebfb9b471aa73afbede0cb6fee P 2aeda88e2ab0477ba816f641cf47d07d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:26:37.102676 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 6523
I20260808 22:26:37.115882 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 6766
I20260808 22:26:37.127813 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 6136
2026-08-08T22:26:37Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2577 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:26:37 dist-test-slave-7f01 krb5kdc[7082](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:26:37 dist-test-slave-7f01 krb5kdc[7082](info): set up 2 sockets
Aug 08 22:26:37 dist-test-slave-7f01 krb5kdc[7082](info): commencing operation
krb5kdc: starting...
W20260808 22:26:38.402073 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.248s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:26:38 dist-test-slave-7f01 krb5kdc[7082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:26:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:26:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.11.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
I20260808 22:26:38.527029 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:39505
--webserver_interface=127.5.11.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:42725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:39505
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:26:38.638032 7103 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:26:38.638340 7103 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:26:38.638417 7103 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:26:38.642352 7103 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:26:38.642437 7103 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 22:26:38.642517 7103 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:26:38.642577 7103 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:26:38.642622 7103 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:26:38.642659 7103 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:26:38.649030 7103 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:42725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:39505
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:39505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7103
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:26:38.650141 7103 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:26:38.651211 7103 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:26:38.659752 7111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:38.659981 7103 server_base.cc:1034] running on GCE node
W20260808 22:26:38.659823 7114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:38.659929 7112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:38.660701 7103 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:26:38.661818 7103 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:26:38.662984 7103 hybrid_clock.cc:648] HybridClock initialized: now 1786227998662954 us; error 42 us; skew 500 ppm
Aug 08 22:26:38 dist-test-slave-7f01 krb5kdc[7082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227998, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:26:38.665777 7103 init.cc:377] Logged in from keytab as kudu/127.5.11.254@KRBTEST.COM (short username kudu)
I20260808 22:26:38.666639 7103 webserver.cc:460] Webserver started at http://127.5.11.254:42215/ using document root <none> and password file <none>
I20260808 22:26:38.667212 7103 fs_manager.cc:359] Metadata directory not provided
I20260808 22:26:38.667305 7103 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:26:38.667541 7103 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:26:38.669615 7103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "576de68957c746e1b39b808dee28cac5"
format_stamp: "Formatted at 2026-08-08 22:26:38 on dist-test-slave-7f01"
server_key: "6a51313ef6a125bfb64a24f40e694471"
server_key_iv: "e5485e568ae2ed9e1d8acec25e4153e4"
server_key_version: "encryptionkey@0"
I20260808 22:26:38.670213 7103 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "576de68957c746e1b39b808dee28cac5"
format_stamp: "Formatted at 2026-08-08 22:26:38 on dist-test-slave-7f01"
server_key: "6a51313ef6a125bfb64a24f40e694471"
server_key_iv: "e5485e568ae2ed9e1d8acec25e4153e4"
server_key_version: "encryptionkey@0"
I20260808 22:26:38.673934 7103 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 22:26:38.676313 7125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:38.677469 7103 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:26:38.677613 7103 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "576de68957c746e1b39b808dee28cac5"
format_stamp: "Formatted at 2026-08-08 22:26:38 on dist-test-slave-7f01"
server_key: "6a51313ef6a125bfb64a24f40e694471"
server_key_iv: "e5485e568ae2ed9e1d8acec25e4153e4"
server_key_version: "encryptionkey@0"
I20260808 22:26:38.677723 7103 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:26:38.699201 7103 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:26:38.701968 7103 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:26:38.702175 7103 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:26:38.710232 7103 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:39505
I20260808 22:26:38.710248 7235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:39505 every 8 connection(s)
I20260808 22:26:38.711153 7103 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:26:38.713104 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 7103
I20260808 22:26:38.713223 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:26:38.713521 5167 external_mini_cluster.cc:1428] Setting key 407b1b14dc8b0f959c600ede24436e5b
I20260808 22:26:38.714368 7236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:38.721374 7236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 576de68957c746e1b39b808dee28cac5: Bootstrap starting.
Aug 08 22:26:38 dist-test-slave-7f01 krb5kdc[7082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227998, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:26:38.724185 7236 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 576de68957c746e1b39b808dee28cac5: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:38.725232 7236 log.cc:824] T 00000000000000000000000000000000 P 576de68957c746e1b39b808dee28cac5: Log is configured to *not* fsync() on all Append() calls
I20260808 22:26:38.727614 7236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 576de68957c746e1b39b808dee28cac5: No bootstrap required, opened a new log
I20260808 22:26:38.729377 7240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:38.714854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56726 (local address 127.5.11.254:39505)
0808 22:26:38.715262 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:38.715281 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:38.715324 (+ 43us) server_negotiation.cc:407] Connection header received
0808 22:26:38.716218 (+ 894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:38.716243 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:38.716660 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:38.717033 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:38.717705 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:38.719188 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:38.720156 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:38.720845 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:38.721009 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:38.723760 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:38.723768 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:38.723783 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:38.723815 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:38.726427 (+ 2612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:38.726868 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:38.726874 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:38.726880 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:38.727022 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:38.727285 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:38.727291 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:38.727295 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:38.727839 (+ 544us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:38.728046 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:38.728421 (+ 375us) server_negotiation.cc:299] Negotiation successful
0808 22:26:38.728681 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":260,"thread_start_us":101,"threads_started":1}
I20260808 22:26:38.730950 7236 raft_consensus.cc:348] T 00000000000000000000000000000000 P 576de68957c746e1b39b808dee28cac5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "576de68957c746e1b39b808dee28cac5" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39505 } }
I20260808 22:26:38.731220 7236 raft_consensus.cc:374] T 00000000000000000000000000000000 P 576de68957c746e1b39b808dee28cac5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:38.731283 7236 raft_consensus.cc:729] T 00000000000000000000000000000000 P 576de68957c746e1b39b808dee28cac5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 576de68957c746e1b39b808dee28cac5, State: Initialized, Role: FOLLOWER
I20260808 22:26:38.731765 7236 consensus_queue.cc:260] T 00000000000000000000000000000000 P 576de68957c746e1b39b808dee28cac5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "576de68957c746e1b39b808dee28cac5" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39505 } }
I20260808 22:26:38.731915 7236 raft_consensus.cc:388] T 00000000000000000000000000000000 P 576de68957c746e1b39b808dee28cac5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:26:38.731964 7236 raft_consensus.cc:482] T 00000000000000000000000000000000 P 576de68957c746e1b39b808dee28cac5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:26:38.732044 7236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 576de68957c746e1b39b808dee28cac5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:38.732852 7236 raft_consensus.cc:504] T 00000000000000000000000000000000 P 576de68957c746e1b39b808dee28cac5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "576de68957c746e1b39b808dee28cac5" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39505 } }
I20260808 22:26:38.733144 7236 leader_election.cc:302] T 00000000000000000000000000000000 P 576de68957c746e1b39b808dee28cac5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 576de68957c746e1b39b808dee28cac5; no voters:
I20260808 22:26:38.733383 7236 leader_election.cc:288] T 00000000000000000000000000000000 P 576de68957c746e1b39b808dee28cac5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:26:38.733489 7247 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 576de68957c746e1b39b808dee28cac5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:26:38.733767 7247 raft_consensus.cc:686] T 00000000000000000000000000000000 P 576de68957c746e1b39b808dee28cac5 [term 1 LEADER]: Becoming Leader. State: Replica: 576de68957c746e1b39b808dee28cac5, State: Running, Role: LEADER
I20260808 22:26:38.734135 7247 consensus_queue.cc:237] T 00000000000000000000000000000000 P 576de68957c746e1b39b808dee28cac5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "576de68957c746e1b39b808dee28cac5" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39505 } }
I20260808 22:26:38.734966 7236 sys_catalog.cc:564] T 00000000000000000000000000000000 P 576de68957c746e1b39b808dee28cac5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:26:38.736020 7252 sys_catalog.cc:455] T 00000000000000000000000000000000 P 576de68957c746e1b39b808dee28cac5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 576de68957c746e1b39b808dee28cac5. Latest consensus state: current_term: 1 leader_uuid: "576de68957c746e1b39b808dee28cac5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "576de68957c746e1b39b808dee28cac5" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39505 } } }
I20260808 22:26:38.736152 7252 sys_catalog.cc:458] T 00000000000000000000000000000000 P 576de68957c746e1b39b808dee28cac5 [sys.catalog]: This master's current role is: LEADER
I20260808 22:26:38.736385 7249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 576de68957c746e1b39b808dee28cac5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "576de68957c746e1b39b808dee28cac5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "576de68957c746e1b39b808dee28cac5" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39505 } } }
I20260808 22:26:38.736505 7249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 576de68957c746e1b39b808dee28cac5 [sys.catalog]: This master's current role is: LEADER
I20260808 22:26:38.736552 7266 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:26:38.739923 7266 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:26:38.745719 7266 catalog_manager.cc:1347] Generated new cluster ID: 9001a7ad6a034f5e989d9a8f4f07deb0
I20260808 22:26:38.745790 7266 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:26:38.765565 7266 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:26:38.766413 7266 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:26:38.775362 7266 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 576de68957c746e1b39b808dee28cac5: Generated new TSK 0
I20260808 22:26:38.776055 7266 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.11.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab.
Entry for principal kudu/127.5.11.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab.
I20260808 22:26:38.813030 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.193:0
--local_ip_for_outbound_sockets=127.5.11.193
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:39505
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:42725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:26:38.928436 7289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:26:38.928809 7289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:26:38.928901 7289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:26:38.932858 7289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:26:38.932938 7289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:26:38.933008 7289 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 22:26:38.933080 7289 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.193
I20260808 22:26:38.938396 7289 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:42725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.11.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.11.254:39505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.193
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:26:38.939392 7289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:26:38.940430 7289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:26:38.949720 7298 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:38.949705 7297 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:38.949723 7300 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:38.950034 7289 server_base.cc:1034] running on GCE node
I20260808 22:26:38.950395 7289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:26:38.950979 7289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:26:38.952172 7289 hybrid_clock.cc:648] HybridClock initialized: now 1786227998952165 us; error 30 us; skew 500 ppm
Aug 08 22:26:38 dist-test-slave-7f01 krb5kdc[7082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227998, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:26:38.955124 7289 init.cc:377] Logged in from keytab as kudu/127.5.11.193@KRBTEST.COM (short username kudu)
I20260808 22:26:38.956029 7289 webserver.cc:460] Webserver started at http://127.5.11.193:40571/ using document root <none> and password file <none>
I20260808 22:26:38.956611 7289 fs_manager.cc:359] Metadata directory not provided
I20260808 22:26:38.956712 7289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:26:38.956952 7289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:26:38.958942 7289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-0/data/instance:
uuid: "34054211be954fcc9530b6b4ebc0ac4e"
format_stamp: "Formatted at 2026-08-08 22:26:38 on dist-test-slave-7f01"
server_key: "40f795c54cbf707fdf8fbb4aefd48cdf"
server_key_iv: "cf560fbc337b7aef0d5d6dcb47bdcd35"
server_key_version: "encryptionkey@0"
I20260808 22:26:38.959522 7289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance:
uuid: "34054211be954fcc9530b6b4ebc0ac4e"
format_stamp: "Formatted at 2026-08-08 22:26:38 on dist-test-slave-7f01"
server_key: "40f795c54cbf707fdf8fbb4aefd48cdf"
server_key_iv: "cf560fbc337b7aef0d5d6dcb47bdcd35"
server_key_version: "encryptionkey@0"
I20260808 22:26:38.963214 7289 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 22:26:38.965655 7314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:38.966850 7289 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:26:38.966993 7289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-0/wal
uuid: "34054211be954fcc9530b6b4ebc0ac4e"
format_stamp: "Formatted at 2026-08-08 22:26:38 on dist-test-slave-7f01"
server_key: "40f795c54cbf707fdf8fbb4aefd48cdf"
server_key_iv: "cf560fbc337b7aef0d5d6dcb47bdcd35"
server_key_version: "encryptionkey@0"
I20260808 22:26:38.967105 7289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:26:38.987921 7289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:26:38.990712 7289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:26:38.990921 7289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:26:38.992735 7289 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:26:38.992803 7289 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:38.992862 7289 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:26:38.992900 7289 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 22:26:38 dist-test-slave-7f01 krb5kdc[7082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227998, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:26:39.004319 7289 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.193:44039
I20260808 22:26:39.004396 7527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.193:44039 every 8 connection(s)
I20260808 22:26:39.005420 7289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
I20260808 22:26:39.006887 7240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:38.993179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:33077 (local address 127.5.11.254:39505)
0808 22:26:38.993308 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:38.993311 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:38.994336 (+ 1025us) server_negotiation.cc:407] Connection header received
0808 22:26:38.995346 (+ 1010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:38.995350 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:38.995414 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:38.995517 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:38.997196 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:38.998107 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:38.999689 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.000033 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.000173 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:39.002962 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:39.002966 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:39.002969 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:39.003005 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:39.004697 (+ 1692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:39.005192 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:39.005196 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:39.005198 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:39.005297 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:39.005695 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:39.005699 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:39.005701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:39.005965 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:39.006078 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:39.006644 (+ 566us) server_negotiation.cc:299] Negotiation successful
0808 22:26:39.006752 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 22:26:39.007620 7330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:38.993722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39505 (local address 127.5.11.193:33077)
0808 22:26:38.994189 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:38.994231 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:38.995066 (+ 835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:38.995722 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:38.995731 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:38.996310 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:38.997038 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:38.997056 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:38.998255 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:38.998259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:38.999503 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:38.999513 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.000223 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.000227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:39.000473 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:39.001134 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:26:39.001155 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:39.002795 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:39.004810 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:39.004815 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:39.004826 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:39.005082 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:39.005397 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:39.005399 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:39.005401 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:39.005597 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:39.006140 (+ 543us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:39.006149 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:39.006417 (+ 268us) client_negotiation.cc:769] Sending connection context
0808 22:26:39.006576 (+ 159us) client_negotiation.cc:240] Negotiation successful
0808 22:26:39.006823 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":128,"threads_started":1}
I20260808 22:26:39.008985 7529 heartbeater.cc:347] Connected to a master server at 127.5.11.254:39505
I20260808 22:26:39.009152 7529 heartbeater.cc:464] Registering TS with master...
I20260808 22:26:39.009629 7529 heartbeater.cc:511] Master 127.5.11.254:39505 requested a full tablet report, sending...
I20260808 22:26:39.009964 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 7289
I20260808 22:26:39.010088 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance
I20260808 22:26:39.010390 5167 external_mini_cluster.cc:1428] Setting key 6addbfef66955a55f5a59160c5fea6f5
I20260808 22:26:39.011278 7158 ts_manager.cc:194] Registered new tserver with Master: 34054211be954fcc9530b6b4ebc0ac4e (127.5.11.193:44039)
I20260808 22:26:39.012698 7158 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.193@KRBTEST.COM'} at 127.5.11.193:33077
I20260808 22:26:39.019009 7537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:39.011213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39505 (local address 127.5.11.193:33723)
0808 22:26:39.011541 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:39.011555 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:39.011687 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:39.011976 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:39.011979 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:39.012176 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:39.012306 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:39.012314 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.013266 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.013272 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:39.014216 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:39.014226 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.014675 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.014679 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:39.014774 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:39.015255 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:26:39.015275 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:39.015587 (+ 312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:39.017567 (+ 1980us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:39.017570 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:39.017571 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:39.017769 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:39.018103 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:39.018106 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:39.018107 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:39.018187 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:39.018616 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:39.018619 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:39.018689 (+ 70us) client_negotiation.cc:769] Sending connection context
0808 22:26:39.018774 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 22:26:39.018869 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":3,"thread_start_us":97,"threads_started":1}
I20260808 22:26:39.019356 7240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:39.011284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:33723 (local address 127.5.11.254:39505)
0808 22:26:39.011392 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:39.011396 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:39.011620 (+ 224us) server_negotiation.cc:407] Connection header received
0808 22:26:39.011808 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:39.011811 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:39.011850 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:39.011929 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:39.012439 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.013120 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.014342 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.014558 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.014669 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:39.015765 (+ 1096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:39.015773 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:39.015778 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:39.015810 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:39.017450 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:39.017879 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:39.017885 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:39.017888 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:39.017997 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:39.018288 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:39.018292 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:39.018293 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:39.018516 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:39.018627 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:39.018852 (+ 225us) server_negotiation.cc:299] Negotiation successful
0808 22:26:39.018981 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
WARNING: no policy specified for kudu/127.5.11.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab.
Entry for principal kudu/127.5.11.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab.
I20260808 22:26:39.040664 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.194:0
--local_ip_for_outbound_sockets=127.5.11.194
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:39505
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:42725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:26:39.150540 7543 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:26:39.150770 7543 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:26:39.150820 7543 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:26:39.154773 7543 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:26:39.154829 7543 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:26:39.154865 7543 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 22:26:39.154893 7543 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.194
I20260808 22:26:39.159842 7543 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:42725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.11.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.11.254:39505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7543
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.194
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:26:39.160638 7543 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:26:39.161527 7543 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:26:39.170634 7550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:39.170627 7551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:39.170627 7553 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:39.170914 7543 server_base.cc:1034] running on GCE node
I20260808 22:26:39.171391 7543 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:26:39.172039 7543 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:26:39.173219 7543 hybrid_clock.cc:648] HybridClock initialized: now 1786227999173206 us; error 23 us; skew 500 ppm
Aug 08 22:26:39 dist-test-slave-7f01 krb5kdc[7082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227999, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:26:39.179178 7543 init.cc:377] Logged in from keytab as kudu/127.5.11.194@KRBTEST.COM (short username kudu)
I20260808 22:26:39.180050 7543 webserver.cc:460] Webserver started at http://127.5.11.194:43357/ using document root <none> and password file <none>
I20260808 22:26:39.180598 7543 fs_manager.cc:359] Metadata directory not provided
I20260808 22:26:39.180687 7543 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:26:39.180933 7543 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:26:39.182897 7543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-1/data/instance:
uuid: "570f0cfa261644d98997a916932ad7df"
format_stamp: "Formatted at 2026-08-08 22:26:39 on dist-test-slave-7f01"
server_key: "79fb72f0bcf3b1be69f6f98e3f2fa8a3"
server_key_iv: "df0a0f00c34db59013e2c2e5303da677"
server_key_version: "encryptionkey@0"
I20260808 22:26:39.183454 7543 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance:
uuid: "570f0cfa261644d98997a916932ad7df"
format_stamp: "Formatted at 2026-08-08 22:26:39 on dist-test-slave-7f01"
server_key: "79fb72f0bcf3b1be69f6f98e3f2fa8a3"
server_key_iv: "df0a0f00c34db59013e2c2e5303da677"
server_key_version: "encryptionkey@0"
I20260808 22:26:39.186962 7543 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 22:26:39.189325 7572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:39.190485 7543 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:26:39.190627 7543 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-1/wal
uuid: "570f0cfa261644d98997a916932ad7df"
format_stamp: "Formatted at 2026-08-08 22:26:39 on dist-test-slave-7f01"
server_key: "79fb72f0bcf3b1be69f6f98e3f2fa8a3"
server_key_iv: "df0a0f00c34db59013e2c2e5303da677"
server_key_version: "encryptionkey@0"
I20260808 22:26:39.190734 7543 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:26:39.200143 7543 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:26:39.203629 7543 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:26:39.203873 7543 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:26:39.205802 7543 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:26:39.205893 7543 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:39.205958 7543 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:26:39.205996 7543 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 22:26:39 dist-test-slave-7f01 krb5kdc[7082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227999, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:26:39.220181 7543 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.194:39773
I20260808 22:26:39.220217 7777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.194:39773 every 8 connection(s)
I20260808 22:26:39.221305 7543 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
I20260808 22:26:39.222924 7240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:39.206575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:60393 (local address 127.5.11.254:39505)
0808 22:26:39.206782 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:39.206815 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:39.209664 (+ 2849us) server_negotiation.cc:407] Connection header received
0808 22:26:39.209729 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:39.209734 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:39.209796 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:39.209909 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:39.211617 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.212424 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.213948 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.214226 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.214341 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:39.218629 (+ 4288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:39.218634 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:39.218637 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:39.218673 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:39.220562 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:39.221174 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:39.221177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:39.221179 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:39.221277 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:39.221700 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:39.221707 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:39.221711 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:39.221936 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:39.222058 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:39.222625 (+ 567us) server_negotiation.cc:299] Negotiation successful
0808 22:26:39.222750 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 22:26:39.223636 7585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:39.207003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39505 (local address 127.5.11.194:60393)
0808 22:26:39.207548 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:39.207595 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:39.208661 (+ 1066us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:39.210183 (+ 1522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:39.210194 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:39.210687 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:39.211389 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:39.211414 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.212581 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.212587 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:39.213812 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:39.213822 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.214547 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.214552 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:39.214796 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:39.215480 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:26:39.215518 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:39.218389 (+ 2871us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:39.220693 (+ 2304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:39.220698 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:39.220711 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:39.221051 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:39.221393 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:39.221396 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:39.221398 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:39.221570 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:39.222058 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:39.222065 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:39.222363 (+ 298us) client_negotiation.cc:769] Sending connection context
0808 22:26:39.222578 (+ 215us) client_negotiation.cc:240] Negotiation successful
0808 22:26:39.222879 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":157,"threads_started":1}
I20260808 22:26:39.227211 7778 heartbeater.cc:347] Connected to a master server at 127.5.11.254:39505
I20260808 22:26:39.227425 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 7543
I20260808 22:26:39.227456 7778 heartbeater.cc:464] Registering TS with master...
I20260808 22:26:39.227563 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance
I20260808 22:26:39.227926 5167 external_mini_cluster.cc:1428] Setting key 53d158da96d99b9443dcd3a415058289
I20260808 22:26:39.228461 7778 heartbeater.cc:511] Master 127.5.11.254:39505 requested a full tablet report, sending...
I20260808 22:26:39.230203 7159 ts_manager.cc:194] Registered new tserver with Master: 570f0cfa261644d98997a916932ad7df (127.5.11.194:39773)
I20260808 22:26:39.231185 7159 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.194@KRBTEST.COM'} at 127.5.11.194:60393
I20260808 22:26:39.241953 7791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:39.231631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39505 (local address 127.5.11.194:41329)
0808 22:26:39.232713 (+ 1082us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:39.232736 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:39.232900 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:39.233211 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:39.233214 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:39.233434 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:39.233539 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:39.233545 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.234572 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.234574 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:39.235629 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:39.235647 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.236343 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.236349 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:39.236513 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:39.237107 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:26:39.237127 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:39.237600 (+ 473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:39.239870 (+ 2270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:39.239874 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:39.239876 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:39.240154 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:39.240519 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:39.240523 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:39.240525 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:39.240635 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:39.241287 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:39.241293 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:39.241397 (+ 104us) client_negotiation.cc:769] Sending connection context
0808 22:26:39.241527 (+ 130us) client_negotiation.cc:240] Negotiation successful
0808 22:26:39.241684 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":990,"thread_start_us":894,"threads_started":1}
I20260808 22:26:39.241962 7240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:39.231368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:41329 (local address 127.5.11.254:39505)
0808 22:26:39.231499 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:39.231506 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:39.232815 (+ 1309us) server_negotiation.cc:407] Connection header received
0808 22:26:39.233014 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:39.233020 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:39.233077 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:39.233161 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:39.233662 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.234445 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.235807 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.236202 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.236310 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:39.237832 (+ 1522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:39.237837 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:39.237839 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:39.237869 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:39.239722 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:39.240277 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:39.240283 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:39.240285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:39.240392 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:39.240808 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:39.240813 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:39.240816 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:39.241157 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:39.241276 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:39.241634 (+ 358us) server_negotiation.cc:299] Negotiation successful
0808 22:26:39.241765 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
WARNING: no policy specified for kudu/127.5.11.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab.
Entry for principal kudu/127.5.11.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab.
I20260808 22:26:39.263060 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.195:0
--local_ip_for_outbound_sockets=127.5.11.195
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:39505
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:42725
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:26:39.376463 7800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:26:39.376760 7800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:26:39.376837 7800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:26:39.380877 7800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:26:39.380960 7800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:26:39.381028 7800 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 22:26:39.381105 7800 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.195
I20260808 22:26:39.386402 7800 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:42725
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.11.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.11.254:39505
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.195
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:26:39.387497 7800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:26:39.388556 7800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:26:39.398208 7816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:39.398257 7813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:39.398209 7808 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:39.398506 7800 server_base.cc:1034] running on GCE node
I20260808 22:26:39.399024 7800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:26:39.399680 7800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:26:39.400888 7800 hybrid_clock.cc:648] HybridClock initialized: now 1786227999400874 us; error 24 us; skew 500 ppm
Aug 08 22:26:39 dist-test-slave-7f01 krb5kdc[7082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227999, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:26:39.403844 7800 init.cc:377] Logged in from keytab as kudu/127.5.11.195@KRBTEST.COM (short username kudu)
I20260808 22:26:39.404675 7800 webserver.cc:460] Webserver started at http://127.5.11.195:33893/ using document root <none> and password file <none>
I20260808 22:26:39.405268 7800 fs_manager.cc:359] Metadata directory not provided
I20260808 22:26:39.405366 7800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:26:39.405601 7800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:26:39.407618 7800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-2/data/instance:
uuid: "7a1c2468f20444faa40177d645c9a1ca"
format_stamp: "Formatted at 2026-08-08 22:26:39 on dist-test-slave-7f01"
server_key: "dbcb3eb771c33171b95fa54fb6f1be4f"
server_key_iv: "82857f6d6785417b6deb6f7499be48d8"
server_key_version: "encryptionkey@0"
I20260808 22:26:39.408253 7800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance:
uuid: "7a1c2468f20444faa40177d645c9a1ca"
format_stamp: "Formatted at 2026-08-08 22:26:39 on dist-test-slave-7f01"
server_key: "dbcb3eb771c33171b95fa54fb6f1be4f"
server_key_iv: "82857f6d6785417b6deb6f7499be48d8"
server_key_version: "encryptionkey@0"
I20260808 22:26:39.412110 7800 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 22:26:39.414923 7828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:39.416193 7800 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:26:39.416291 7800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-2/wal
uuid: "7a1c2468f20444faa40177d645c9a1ca"
format_stamp: "Formatted at 2026-08-08 22:26:39 on dist-test-slave-7f01"
server_key: "dbcb3eb771c33171b95fa54fb6f1be4f"
server_key_iv: "82857f6d6785417b6deb6f7499be48d8"
server_key_version: "encryptionkey@0"
I20260808 22:26:39.416373 7800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:26:39.429279 7800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:26:39.432327 7800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:26:39.432475 7800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:26:39.433851 7800 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:26:39.433929 7800 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:39.434001 7800 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:26:39.434037 7800 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 22:26:39 dist-test-slave-7f01 krb5kdc[7082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227999, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:26:39.445660 7800 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.195:38875
I20260808 22:26:39.445788 8033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.195:38875 every 8 connection(s)
I20260808 22:26:39.446826 7800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
I20260808 22:26:39.448873 7240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:39.434791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:38551 (local address 127.5.11.254:39505)
0808 22:26:39.434928 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:39.434932 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:39.436123 (+ 1191us) server_negotiation.cc:407] Connection header received
0808 22:26:39.437299 (+ 1176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:39.437304 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:39.437378 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:39.437485 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:39.439078 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.439926 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.441064 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.441295 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.441432 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:39.444794 (+ 3362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:39.444798 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:39.444800 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:39.444849 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:39.446625 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:39.447226 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:39.447231 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:39.447233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:39.447337 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:39.447778 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:39.447782 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:39.447784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:39.447975 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:39.448092 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:39.448577 (+ 485us) server_negotiation.cc:299] Negotiation successful
0808 22:26:39.448700 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 22:26:39.449611 7841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:39.435360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39505 (local address 127.5.11.195:38551)
0808 22:26:39.435958 (+ 598us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:39.436001 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:39.437044 (+ 1043us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:39.437714 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:39.437726 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:39.438246 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:39.438889 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:39.438934 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.440057 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.440060 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:39.440931 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:39.440942 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.441648 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.441652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:39.441875 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:39.442564 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:26:39.442582 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:39.444620 (+ 2038us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:39.446753 (+ 2133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:39.446757 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:39.446780 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:39.447115 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:39.447450 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:39.447453 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:39.447455 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:39.447633 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:39.448090 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:39.448097 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:39.448338 (+ 241us) client_negotiation.cc:769] Sending connection context
0808 22:26:39.448529 (+ 191us) client_negotiation.cc:240] Negotiation successful
0808 22:26:39.448844 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":150,"threads_started":1}
I20260808 22:26:39.450157 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 7800
I20260808 22:26:39.450291 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance
I20260808 22:26:39.450573 5167 external_mini_cluster.cc:1428] Setting key f1e1149d5be91b5b93758f659cdb9465
I20260808 22:26:39.450955 8035 heartbeater.cc:347] Connected to a master server at 127.5.11.254:39505
I20260808 22:26:39.451117 8035 heartbeater.cc:464] Registering TS with master...
I20260808 22:26:39.451704 8035 heartbeater.cc:511] Master 127.5.11.254:39505 requested a full tablet report, sending...
I20260808 22:26:39.452841 7159 ts_manager.cc:194] Registered new tserver with Master: 7a1c2468f20444faa40177d645c9a1ca (127.5.11.195:38875)
I20260808 22:26:39.453568 7159 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.195@KRBTEST.COM'} at 127.5.11.195:38551
I20260808 22:26:39.460207 7240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:39.453324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:35033 (local address 127.5.11.254:39505)
0808 22:26:39.453463 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:39.453467 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:39.453689 (+ 222us) server_negotiation.cc:407] Connection header received
0808 22:26:39.453800 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:39.453826 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:39.453880 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:39.453957 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:39.454439 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.455162 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.456041 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.456245 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.456329 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:39.457355 (+ 1026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:39.457358 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:39.457360 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:39.457382 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:39.458856 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:39.459217 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:39.459220 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:39.459221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:39.459295 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:39.459527 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:39.459529 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:39.459531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:39.459685 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:39.459808 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:39.460017 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 22:26:39.460106 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 22:26:39.460207 8047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:39.453262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39505 (local address 127.5.11.195:35033)
0808 22:26:39.453595 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:39.453610 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:39.453710 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:39.453996 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:39.454000 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:39.454212 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:39.454322 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:39.454327 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.455297 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.455300 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:39.455935 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:39.455944 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.456342 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.456344 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:39.456412 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:39.456803 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:26:39.456814 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:39.457224 (+ 410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:39.458946 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:39.458948 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:39.458950 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:39.459135 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:39.459373 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:39.459375 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:39.459377 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:39.459450 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:39.459815 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:39.459818 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:39.459892 (+ 74us) client_negotiation.cc:769] Sending connection context
0808 22:26:39.459974 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 22:26:39.460086 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":147,"threads_started":1}
I20260808 22:26:39.464563 5167 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 22:26:39 dist-test-slave-7f01 krb5kdc[7082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227999, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 22:26:39 dist-test-slave-7f01 krb5kdc[7082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227999, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:26:39.491685 7240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:39.483381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56736 (local address 127.5.11.254:39505)
0808 22:26:39.483513 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:39.483517 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:39.483527 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:26:39.483557 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:39.483559 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:39.483592 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:39.483679 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:39.484403 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.484907 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.485741 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.485964 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.486070 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:39.488502 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:39.488507 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:39.488514 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:39.488537 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:39.489989 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:39.490441 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:39.490445 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:39.490446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:39.490523 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:39.490815 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:39.490819 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:39.490821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:39.490998 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:39.491141 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:39.491402 (+ 261us) server_negotiation.cc:299] Negotiation successful
0808 22:26:39.491525 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 22:26:39.494859 7159 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56736:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:26:39.497929 7159 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:26:39.512245 8070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:39.506209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:44039 (local address 127.0.0.1:60480)
0808 22:26:39.506526 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:39.506571 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:39.507469 (+ 898us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:39.508335 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:39.508338 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:39.508359 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:26:39.508557 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:39.508567 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.509713 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.509716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:39.510804 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:39.510815 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.511672 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.511676 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:39.511821 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:39.511867 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 22:26:39.511982 (+ 115us) client_negotiation.cc:240] Negotiation successful
0808 22:26:39.512096 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":71,"spinlock_wait_cycles":8064,"thread_start_us":82,"threads_started":1}
I20260808 22:26:39.512269 8077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:39.506961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60480 (local address 127.5.11.193:44039)
0808 22:26:39.507475 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:39.507504 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:39.507534 (+ 30us) server_negotiation.cc:407] Connection header received
0808 22:26:39.507625 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:39.507630 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:39.507874 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:39.508202 (+ 328us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:26:39.508682 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.509587 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.510936 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.511547 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.511656 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:39.511830 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:39.512032 (+ 202us) server_negotiation.cc:299] Negotiation successful
0808 22:26:39.512132 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":107,"threads_started":1}
I20260808 22:26:39.513067 8073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:39.506301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:38875 (local address 127.0.0.1:48086)
0808 22:26:39.507441 (+ 1140us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:39.507455 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:39.507539 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:39.508181 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:39.508187 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:39.508210 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:26:39.508351 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:39.508359 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.509894 (+ 1535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.509898 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:39.511345 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:39.511357 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.512654 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.512659 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:39.512747 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:39.512774 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 22:26:39.512847 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 22:26:39.512915 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1076,"mutex_wait_us":128,"thread_start_us":114,"threads_started":1}
I20260808 22:26:39.513211 8071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:39.506592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:39773 (local address 127.0.0.1:41296)
0808 22:26:39.507706 (+ 1114us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:39.507716 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:39.507851 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:39.508345 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:39.508347 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:39.508372 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:26:39.508485 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:39.508493 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.509894 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.509898 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:39.511345 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:39.511357 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.512824 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.512828 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:39.512915 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:39.512943 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 22:26:39.513017 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 22:26:39.513087 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1057,"mutex_wait_us":135,"spinlock_wait_cycles":9984,"thread_start_us":96,"threads_started":1}
I20260808 22:26:39.513486 8078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:39.507018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48086 (local address 127.5.11.195:38875)
0808 22:26:39.507711 (+ 693us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:39.507722 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:39.507765 (+ 43us) server_negotiation.cc:407] Connection header received
0808 22:26:39.507837 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:39.507842 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:39.508031 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:39.508187 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:26:39.508492 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.509736 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.511484 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.512309 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.512421 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:39.512575 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:39.513310 (+ 735us) server_negotiation.cc:299] Negotiation successful
0808 22:26:39.513385 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":127,"threads_started":1}
I20260808 22:26:39.513671 8080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:39.507037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41296 (local address 127.5.11.194:39773)
0808 22:26:39.507525 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:39.507536 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:39.507839 (+ 303us) server_negotiation.cc:407] Connection header received
0808 22:26:39.507987 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:39.507995 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:39.508186 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:39.508356 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:26:39.508606 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.509752 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.511502 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.512485 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.512588 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:39.512747 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:39.513429 (+ 682us) server_negotiation.cc:299] Negotiation successful
0808 22:26:39.513526 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":105,"threads_started":1}
I20260808 22:26:39.515846 7414 tablet_service.cc:1467] Processing CreateTablet for tablet ca7986efe42c4fa79956e4fb00f9fc4a (DEFAULT_TABLE table=test-table [id=3f040f8acd544b2fb2d783e7c5e1d36b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:26:39.516041 7658 tablet_service.cc:1467] Processing CreateTablet for tablet ca7986efe42c4fa79956e4fb00f9fc4a (DEFAULT_TABLE table=test-table [id=3f040f8acd544b2fb2d783e7c5e1d36b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:26:39.516031 7915 tablet_service.cc:1467] Processing CreateTablet for tablet ca7986efe42c4fa79956e4fb00f9fc4a (DEFAULT_TABLE table=test-table [id=3f040f8acd544b2fb2d783e7c5e1d36b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:26:39.516950 7658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca7986efe42c4fa79956e4fb00f9fc4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:39.517035 7915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca7986efe42c4fa79956e4fb00f9fc4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:39.517030 7414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca7986efe42c4fa79956e4fb00f9fc4a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:39.524176 8084 tablet_bootstrap.cc:492] T ca7986efe42c4fa79956e4fb00f9fc4a P 570f0cfa261644d98997a916932ad7df: Bootstrap starting.
I20260808 22:26:39.525162 8085 tablet_bootstrap.cc:492] T ca7986efe42c4fa79956e4fb00f9fc4a P 34054211be954fcc9530b6b4ebc0ac4e: Bootstrap starting.
I20260808 22:26:39.526437 8084 tablet_bootstrap.cc:654] T ca7986efe42c4fa79956e4fb00f9fc4a P 570f0cfa261644d98997a916932ad7df: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:39.526964 8085 tablet_bootstrap.cc:654] T ca7986efe42c4fa79956e4fb00f9fc4a P 34054211be954fcc9530b6b4ebc0ac4e: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:39.527391 8084 log.cc:824] T ca7986efe42c4fa79956e4fb00f9fc4a P 570f0cfa261644d98997a916932ad7df: Log is configured to *not* fsync() on all Append() calls
I20260808 22:26:39.527724 8085 log.cc:824] T ca7986efe42c4fa79956e4fb00f9fc4a P 34054211be954fcc9530b6b4ebc0ac4e: Log is configured to *not* fsync() on all Append() calls
I20260808 22:26:39.528453 8086 tablet_bootstrap.cc:492] T ca7986efe42c4fa79956e4fb00f9fc4a P 7a1c2468f20444faa40177d645c9a1ca: Bootstrap starting.
I20260808 22:26:39.529716 8084 tablet_bootstrap.cc:492] T ca7986efe42c4fa79956e4fb00f9fc4a P 570f0cfa261644d98997a916932ad7df: No bootstrap required, opened a new log
I20260808 22:26:39.529906 8084 ts_tablet_manager.cc:1397] T ca7986efe42c4fa79956e4fb00f9fc4a P 570f0cfa261644d98997a916932ad7df: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 22:26:39.530328 8086 tablet_bootstrap.cc:654] T ca7986efe42c4fa79956e4fb00f9fc4a P 7a1c2468f20444faa40177d645c9a1ca: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:39.530984 8086 log.cc:824] T ca7986efe42c4fa79956e4fb00f9fc4a P 7a1c2468f20444faa40177d645c9a1ca: Log is configured to *not* fsync() on all Append() calls
I20260808 22:26:39.532450 8085 tablet_bootstrap.cc:492] T ca7986efe42c4fa79956e4fb00f9fc4a P 34054211be954fcc9530b6b4ebc0ac4e: No bootstrap required, opened a new log
I20260808 22:26:39.532663 8085 ts_tablet_manager.cc:1397] T ca7986efe42c4fa79956e4fb00f9fc4a P 34054211be954fcc9530b6b4ebc0ac4e: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.003s
I20260808 22:26:39.532940 8086 tablet_bootstrap.cc:492] T ca7986efe42c4fa79956e4fb00f9fc4a P 7a1c2468f20444faa40177d645c9a1ca: No bootstrap required, opened a new log
I20260808 22:26:39.533123 8086 ts_tablet_manager.cc:1397] T ca7986efe42c4fa79956e4fb00f9fc4a P 7a1c2468f20444faa40177d645c9a1ca: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 22:26:39.533586 8084 raft_consensus.cc:348] T ca7986efe42c4fa79956e4fb00f9fc4a P 570f0cfa261644d98997a916932ad7df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } }
I20260808 22:26:39.533855 8084 raft_consensus.cc:374] T ca7986efe42c4fa79956e4fb00f9fc4a P 570f0cfa261644d98997a916932ad7df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:39.533917 8084 raft_consensus.cc:729] T ca7986efe42c4fa79956e4fb00f9fc4a P 570f0cfa261644d98997a916932ad7df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 570f0cfa261644d98997a916932ad7df, State: Initialized, Role: FOLLOWER
I20260808 22:26:39.534382 8084 consensus_queue.cc:260] T ca7986efe42c4fa79956e4fb00f9fc4a P 570f0cfa261644d98997a916932ad7df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } }
I20260808 22:26:39.535392 8084 ts_tablet_manager.cc:1428] T ca7986efe42c4fa79956e4fb00f9fc4a P 570f0cfa261644d98997a916932ad7df: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 22:26:39.535768 7778 heartbeater.cc:503] Master 127.5.11.254:39505 was elected leader, sending a full tablet report...
I20260808 22:26:39.536787 8085 raft_consensus.cc:348] T ca7986efe42c4fa79956e4fb00f9fc4a P 34054211be954fcc9530b6b4ebc0ac4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } }
I20260808 22:26:39.537055 8085 raft_consensus.cc:374] T ca7986efe42c4fa79956e4fb00f9fc4a P 34054211be954fcc9530b6b4ebc0ac4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:39.536897 8086 raft_consensus.cc:348] T ca7986efe42c4fa79956e4fb00f9fc4a P 7a1c2468f20444faa40177d645c9a1ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } }
I20260808 22:26:39.537119 8085 raft_consensus.cc:729] T ca7986efe42c4fa79956e4fb00f9fc4a P 34054211be954fcc9530b6b4ebc0ac4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34054211be954fcc9530b6b4ebc0ac4e, State: Initialized, Role: FOLLOWER
I20260808 22:26:39.537178 8086 raft_consensus.cc:374] T ca7986efe42c4fa79956e4fb00f9fc4a P 7a1c2468f20444faa40177d645c9a1ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:39.537238 8086 raft_consensus.cc:729] T ca7986efe42c4fa79956e4fb00f9fc4a P 7a1c2468f20444faa40177d645c9a1ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a1c2468f20444faa40177d645c9a1ca, State: Initialized, Role: FOLLOWER
I20260808 22:26:39.537575 8085 consensus_queue.cc:260] T ca7986efe42c4fa79956e4fb00f9fc4a P 34054211be954fcc9530b6b4ebc0ac4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } }
I20260808 22:26:39.537655 8086 consensus_queue.cc:260] T ca7986efe42c4fa79956e4fb00f9fc4a P 7a1c2468f20444faa40177d645c9a1ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } }
I20260808 22:26:39.538398 8085 ts_tablet_manager.cc:1428] T ca7986efe42c4fa79956e4fb00f9fc4a P 34054211be954fcc9530b6b4ebc0ac4e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 22:26:39.538631 8086 ts_tablet_manager.cc:1428] T ca7986efe42c4fa79956e4fb00f9fc4a P 7a1c2468f20444faa40177d645c9a1ca: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 22:26:39.538856 7529 heartbeater.cc:503] Master 127.5.11.254:39505 was elected leader, sending a full tablet report...
I20260808 22:26:39.538975 8035 heartbeater.cc:503] Master 127.5.11.254:39505 was elected leader, sending a full tablet report...
I20260808 22:26:39.650276 8091 raft_consensus.cc:482] T ca7986efe42c4fa79956e4fb00f9fc4a P 570f0cfa261644d98997a916932ad7df [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:26:39.650537 8091 raft_consensus.cc:504] T ca7986efe42c4fa79956e4fb00f9fc4a P 570f0cfa261644d98997a916932ad7df [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } }
I20260808 22:26:39.651911 8091 leader_election.cc:288] T ca7986efe42c4fa79956e4fb00f9fc4a P 570f0cfa261644d98997a916932ad7df [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34054211be954fcc9530b6b4ebc0ac4e (127.5.11.193:44039), 7a1c2468f20444faa40177d645c9a1ca (127.5.11.195:38875)
I20260808 22:26:39.656111 8077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:39.652287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:44917 (local address 127.5.11.193:44039)
0808 22:26:39.652404 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:39.652409 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:39.652426 (+ 17us) server_negotiation.cc:407] Connection header received
0808 22:26:39.652582 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:39.652585 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:39.652644 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:39.652742 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:26:39.653076 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.653689 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.654849 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.655433 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.655560 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:39.655615 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:39.655945 (+ 330us) server_negotiation.cc:299] Negotiation successful
0808 22:26:39.656011 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 22:26:39.656116 7585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:39.652143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:44039 (local address 127.5.11.194:44917)
0808 22:26:39.652319 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:39.652338 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:39.652444 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:39.652772 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:39.652778 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:39.652801 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:26:39.652955 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:39.652966 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.653798 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.653801 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:39.654705 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:39.654716 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.655673 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.655677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:39.655808 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:39.655833 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 22:26:39.655910 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 22:26:39.655978 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260808 22:26:39.656836 8100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:39.652157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:38875 (local address 127.5.11.194:38873)
0808 22:26:39.652648 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:39.652662 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:39.652761 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:39.653054 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:39.653058 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:39.653077 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:26:39.653195 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:39.653202 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.654013 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.654016 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:39.655373 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:39.655384 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.656426 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.656435 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:39.656515 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:39.656532 (+ 17us) client_negotiation.cc:769] Sending connection context
0808 22:26:39.656625 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 22:26:39.656691 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"mutex_wait_us":119,"thread_start_us":160,"threads_started":1}
I20260808 22:26:39.656829 8078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:39.652267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:38873 (local address 127.5.11.195:38875)
0808 22:26:39.652392 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:39.652397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:39.652752 (+ 355us) server_negotiation.cc:407] Connection header received
0808 22:26:39.652861 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:39.652864 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:39.652924 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:39.653009 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:26:39.653309 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.653849 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.655533 (+ 1684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.656186 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.656292 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:39.656358 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:39.656650 (+ 292us) server_negotiation.cc:299] Negotiation successful
0808 22:26:39.656725 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 22:26:39.657095 7449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca7986efe42c4fa79956e4fb00f9fc4a" candidate_uuid: "570f0cfa261644d98997a916932ad7df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34054211be954fcc9530b6b4ebc0ac4e" is_pre_election: true
I20260808 22:26:39.657460 7449 raft_consensus.cc:2454] T ca7986efe42c4fa79956e4fb00f9fc4a P 34054211be954fcc9530b6b4ebc0ac4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 570f0cfa261644d98997a916932ad7df in term 0.
I20260808 22:26:39.657581 7954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca7986efe42c4fa79956e4fb00f9fc4a" candidate_uuid: "570f0cfa261644d98997a916932ad7df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a1c2468f20444faa40177d645c9a1ca" is_pre_election: true
I20260808 22:26:39.657866 7954 raft_consensus.cc:2454] T ca7986efe42c4fa79956e4fb00f9fc4a P 7a1c2468f20444faa40177d645c9a1ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 570f0cfa261644d98997a916932ad7df in term 0.
I20260808 22:26:39.657917 7578 leader_election.cc:302] T ca7986efe42c4fa79956e4fb00f9fc4a P 570f0cfa261644d98997a916932ad7df [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34054211be954fcc9530b6b4ebc0ac4e, 570f0cfa261644d98997a916932ad7df; no voters:
I20260808 22:26:39.658167 8091 raft_consensus.cc:2781] T ca7986efe42c4fa79956e4fb00f9fc4a P 570f0cfa261644d98997a916932ad7df [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:26:39.658260 8091 raft_consensus.cc:482] T ca7986efe42c4fa79956e4fb00f9fc4a P 570f0cfa261644d98997a916932ad7df [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:26:39.658305 8091 raft_consensus.cc:3037] T ca7986efe42c4fa79956e4fb00f9fc4a P 570f0cfa261644d98997a916932ad7df [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:39.659353 8091 raft_consensus.cc:504] T ca7986efe42c4fa79956e4fb00f9fc4a P 570f0cfa261644d98997a916932ad7df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } }
I20260808 22:26:39.659682 8091 leader_election.cc:288] T ca7986efe42c4fa79956e4fb00f9fc4a P 570f0cfa261644d98997a916932ad7df [CANDIDATE]: Term 1 election: Requested vote from peers 34054211be954fcc9530b6b4ebc0ac4e (127.5.11.193:44039), 7a1c2468f20444faa40177d645c9a1ca (127.5.11.195:38875)
I20260808 22:26:39.659983 7449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca7986efe42c4fa79956e4fb00f9fc4a" candidate_uuid: "570f0cfa261644d98997a916932ad7df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34054211be954fcc9530b6b4ebc0ac4e"
I20260808 22:26:39.660112 7954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca7986efe42c4fa79956e4fb00f9fc4a" candidate_uuid: "570f0cfa261644d98997a916932ad7df" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a1c2468f20444faa40177d645c9a1ca"
I20260808 22:26:39.660141 7449 raft_consensus.cc:3037] T ca7986efe42c4fa79956e4fb00f9fc4a P 34054211be954fcc9530b6b4ebc0ac4e [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:39.660256 7954 raft_consensus.cc:3037] T ca7986efe42c4fa79956e4fb00f9fc4a P 7a1c2468f20444faa40177d645c9a1ca [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:39.661216 7449 raft_consensus.cc:2454] T ca7986efe42c4fa79956e4fb00f9fc4a P 34054211be954fcc9530b6b4ebc0ac4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 570f0cfa261644d98997a916932ad7df in term 1.
I20260808 22:26:39.661525 7578 leader_election.cc:302] T ca7986efe42c4fa79956e4fb00f9fc4a P 570f0cfa261644d98997a916932ad7df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34054211be954fcc9530b6b4ebc0ac4e, 570f0cfa261644d98997a916932ad7df; no voters:
I20260808 22:26:39.661688 8091 raft_consensus.cc:2781] T ca7986efe42c4fa79956e4fb00f9fc4a P 570f0cfa261644d98997a916932ad7df [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:26:39.661826 7954 raft_consensus.cc:2454] T ca7986efe42c4fa79956e4fb00f9fc4a P 7a1c2468f20444faa40177d645c9a1ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 570f0cfa261644d98997a916932ad7df in term 1.
I20260808 22:26:39.661962 8091 raft_consensus.cc:686] T ca7986efe42c4fa79956e4fb00f9fc4a P 570f0cfa261644d98997a916932ad7df [term 1 LEADER]: Becoming Leader. State: Replica: 570f0cfa261644d98997a916932ad7df, State: Running, Role: LEADER
I20260808 22:26:39.662335 8091 consensus_queue.cc:237] T ca7986efe42c4fa79956e4fb00f9fc4a P 570f0cfa261644d98997a916932ad7df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } }
I20260808 22:26:39.665570 7158 catalog_manager.cc:5495] T ca7986efe42c4fa79956e4fb00f9fc4a P 570f0cfa261644d98997a916932ad7df reported cstate change: term changed from 0 to 1, leader changed from <none> to 570f0cfa261644d98997a916932ad7df (127.5.11.194). New cstate: current_term: 1 leader_uuid: "570f0cfa261644d98997a916932ad7df" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } health_report { overall_health: HEALTHY } } }
W20260808 22:26:39.697868 8037 tablet.cc:2369] T ca7986efe42c4fa79956e4fb00f9fc4a P 7a1c2468f20444faa40177d645c9a1ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:26:39.705893 8080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:39.702385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41300 (local address 127.5.11.194:39773)
0808 22:26:39.702513 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:39.702516 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:39.702530 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:26:39.702686 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:39.702688 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:39.702746 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:39.702833 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:26:39.703076 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.703567 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.704563 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.704787 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.704870 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:39.705180 (+ 310us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:26:39.705575 (+ 395us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:26:39.705665 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:39.705737 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 22:26:39.705786 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 22:26:39.711938 7449 raft_consensus.cc:1261] T ca7986efe42c4fa79956e4fb00f9fc4a P 34054211be954fcc9530b6b4ebc0ac4e [term 1 FOLLOWER]: Refusing update from remote peer 570f0cfa261644d98997a916932ad7df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:26:39.711938 7954 raft_consensus.cc:1261] T ca7986efe42c4fa79956e4fb00f9fc4a P 7a1c2468f20444faa40177d645c9a1ca [term 1 FOLLOWER]: Refusing update from remote peer 570f0cfa261644d98997a916932ad7df: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:26:39.712663 8091 consensus_queue.cc:1035] T ca7986efe42c4fa79956e4fb00f9fc4a P 570f0cfa261644d98997a916932ad7df [LEADER]: Connected to new peer: Peer: permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:26:39.712859 8101 consensus_queue.cc:1035] T ca7986efe42c4fa79956e4fb00f9fc4a P 570f0cfa261644d98997a916932ad7df [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:26:39.723362 8106 mvcc.cc:205] Tried to move back new op lower bound from 7316389886812225536 to 7316389886619590656. Current Snapshot: MvccSnapshot[applied={T|T < 7316389886812225536}]
I20260808 22:26:39.729131 8107 mvcc.cc:205] Tried to move back new op lower bound from 7316389886812225536 to 7316389886619590656. Current Snapshot: MvccSnapshot[applied={T|T < 7316389886812225536}]
I20260808 22:26:39.732439 7158 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:26:39.732695 7158 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:26:39.735140 7158 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 576de68957c746e1b39b808dee28cac5: Sending DeleteTablet for 3 replicas of tablet ca7986efe42c4fa79956e4fb00f9fc4a
I20260808 22:26:39.735917 7414 tablet_service.cc:1514] Processing DeleteTablet for tablet ca7986efe42c4fa79956e4fb00f9fc4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:26:39 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:60480
I20260808 22:26:39.735976 7915 tablet_service.cc:1514] Processing DeleteTablet for tablet ca7986efe42c4fa79956e4fb00f9fc4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:26:39 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:48086
I20260808 22:26:39.736382 8130 tablet_replica.cc:330] T ca7986efe42c4fa79956e4fb00f9fc4a P 34054211be954fcc9530b6b4ebc0ac4e: stopping tablet replica
I20260808 22:26:39.736433 7658 tablet_service.cc:1514] Processing DeleteTablet for tablet ca7986efe42c4fa79956e4fb00f9fc4a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:26:39 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:41296
I20260808 22:26:39.736631 8131 tablet_replica.cc:330] T ca7986efe42c4fa79956e4fb00f9fc4a P 7a1c2468f20444faa40177d645c9a1ca: stopping tablet replica
I20260808 22:26:39.736707 8130 raft_consensus.cc:2229] T ca7986efe42c4fa79956e4fb00f9fc4a P 34054211be954fcc9530b6b4ebc0ac4e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:26:39.737037 8131 raft_consensus.cc:2229] T ca7986efe42c4fa79956e4fb00f9fc4a P 7a1c2468f20444faa40177d645c9a1ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:26:39.737038 8130 raft_consensus.cc:2258] T ca7986efe42c4fa79956e4fb00f9fc4a P 34054211be954fcc9530b6b4ebc0ac4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:26:39.737416 8131 raft_consensus.cc:2258] T ca7986efe42c4fa79956e4fb00f9fc4a P 7a1c2468f20444faa40177d645c9a1ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:26:39.737784 8133 tablet_replica.cc:330] T ca7986efe42c4fa79956e4fb00f9fc4a P 570f0cfa261644d98997a916932ad7df: stopping tablet replica
I20260808 22:26:39.737761 7158 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56736:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 22:26:39.738034 8133 raft_consensus.cc:2229] T ca7986efe42c4fa79956e4fb00f9fc4a P 570f0cfa261644d98997a916932ad7df [term 1 LEADER]: Raft consensus shutting down.
W20260808 22:26:39.738168 7158 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:26:39.738392 8133 raft_consensus.cc:2258] T ca7986efe42c4fa79956e4fb00f9fc4a P 570f0cfa261644d98997a916932ad7df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:26:39.739167 8131 ts_tablet_manager.cc:1905] T ca7986efe42c4fa79956e4fb00f9fc4a P 7a1c2468f20444faa40177d645c9a1ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:26:39.739565 8130 ts_tablet_manager.cc:1905] T ca7986efe42c4fa79956e4fb00f9fc4a P 34054211be954fcc9530b6b4ebc0ac4e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:26:39.742163 8131 ts_tablet_manager.cc:1918] T ca7986efe42c4fa79956e4fb00f9fc4a P 7a1c2468f20444faa40177d645c9a1ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:26:39.742333 8131 log.cc:1196] T ca7986efe42c4fa79956e4fb00f9fc4a P 7a1c2468f20444faa40177d645c9a1ca: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-2/wal/wals/ca7986efe42c4fa79956e4fb00f9fc4a
I20260808 22:26:39.742400 8130 ts_tablet_manager.cc:1918] T ca7986efe42c4fa79956e4fb00f9fc4a P 34054211be954fcc9530b6b4ebc0ac4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:26:39.742789 8131 ts_tablet_manager.cc:1939] T ca7986efe42c4fa79956e4fb00f9fc4a P 7a1c2468f20444faa40177d645c9a1ca: Deleting consensus metadata
I20260808 22:26:39.743788 8133 ts_tablet_manager.cc:1905] T ca7986efe42c4fa79956e4fb00f9fc4a P 570f0cfa261644d98997a916932ad7df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:26:39.743904 7658 tablet_service.cc:1467] Processing CreateTablet for tablet 7bdddf684a8c435290333be03b313b8a (DEFAULT_TABLE table=test-table [id=d42c567d68d54135a5b6aa31c94662a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:26:39.744158 7658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bdddf684a8c435290333be03b313b8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:39.745548 7915 tablet_service.cc:1467] Processing CreateTablet for tablet 7bdddf684a8c435290333be03b313b8a (DEFAULT_TABLE table=test-table [id=d42c567d68d54135a5b6aa31c94662a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:26:39.745817 7915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bdddf684a8c435290333be03b313b8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:39.746467 8133 ts_tablet_manager.cc:1918] T ca7986efe42c4fa79956e4fb00f9fc4a P 570f0cfa261644d98997a916932ad7df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:26:39.746564 8133 log.cc:1196] T ca7986efe42c4fa79956e4fb00f9fc4a P 570f0cfa261644d98997a916932ad7df: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-1/wal/wals/ca7986efe42c4fa79956e4fb00f9fc4a
I20260808 22:26:39.746902 8133 ts_tablet_manager.cc:1939] T ca7986efe42c4fa79956e4fb00f9fc4a P 570f0cfa261644d98997a916932ad7df: Deleting consensus metadata
I20260808 22:26:39.742522 8130 log.cc:1196] T ca7986efe42c4fa79956e4fb00f9fc4a P 34054211be954fcc9530b6b4ebc0ac4e: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-0/wal/wals/ca7986efe42c4fa79956e4fb00f9fc4a
I20260808 22:26:39.747622 8130 ts_tablet_manager.cc:1939] T ca7986efe42c4fa79956e4fb00f9fc4a P 34054211be954fcc9530b6b4ebc0ac4e: Deleting consensus metadata
I20260808 22:26:39.748291 8084 tablet_bootstrap.cc:492] T 7bdddf684a8c435290333be03b313b8a P 570f0cfa261644d98997a916932ad7df: Bootstrap starting.
I20260808 22:26:39.748832 8086 tablet_bootstrap.cc:492] T 7bdddf684a8c435290333be03b313b8a P 7a1c2468f20444faa40177d645c9a1ca: Bootstrap starting.
I20260808 22:26:39.749246 8084 tablet_bootstrap.cc:654] T 7bdddf684a8c435290333be03b313b8a P 570f0cfa261644d98997a916932ad7df: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:39.749815 8086 tablet_bootstrap.cc:654] T 7bdddf684a8c435290333be03b313b8a P 7a1c2468f20444faa40177d645c9a1ca: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:39.750533 8084 tablet_bootstrap.cc:492] T 7bdddf684a8c435290333be03b313b8a P 570f0cfa261644d98997a916932ad7df: No bootstrap required, opened a new log
I20260808 22:26:39.750620 8084 ts_tablet_manager.cc:1397] T 7bdddf684a8c435290333be03b313b8a P 570f0cfa261644d98997a916932ad7df: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 22:26:39.750799 7136 catalog_manager.cc:4841] TS 34054211be954fcc9530b6b4ebc0ac4e (127.5.11.193:44039): tablet ca7986efe42c4fa79956e4fb00f9fc4a (table test-table [id=3f040f8acd544b2fb2d783e7c5e1d36b]) successfully deleted
I20260808 22:26:39.751108 8084 raft_consensus.cc:348] T 7bdddf684a8c435290333be03b313b8a P 570f0cfa261644d98997a916932ad7df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } }
I20260808 22:26:39.751220 8084 raft_consensus.cc:374] T 7bdddf684a8c435290333be03b313b8a P 570f0cfa261644d98997a916932ad7df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:39.751185 7414 tablet_service.cc:1467] Processing CreateTablet for tablet 7bdddf684a8c435290333be03b313b8a (DEFAULT_TABLE table=test-table [id=d42c567d68d54135a5b6aa31c94662a1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:26:39.751252 8084 raft_consensus.cc:729] T 7bdddf684a8c435290333be03b313b8a P 570f0cfa261644d98997a916932ad7df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 570f0cfa261644d98997a916932ad7df, State: Initialized, Role: FOLLOWER
I20260808 22:26:39.751384 8084 consensus_queue.cc:260] T 7bdddf684a8c435290333be03b313b8a P 570f0cfa261644d98997a916932ad7df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } }
I20260808 22:26:39.751448 7414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7bdddf684a8c435290333be03b313b8a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:39.752210 7131 catalog_manager.cc:4841] TS 7a1c2468f20444faa40177d645c9a1ca (127.5.11.195:38875): tablet ca7986efe42c4fa79956e4fb00f9fc4a (table test-table [id=3f040f8acd544b2fb2d783e7c5e1d36b]) successfully deleted
I20260808 22:26:39.752439 8086 tablet_bootstrap.cc:492] T 7bdddf684a8c435290333be03b313b8a P 7a1c2468f20444faa40177d645c9a1ca: No bootstrap required, opened a new log
I20260808 22:26:39.752511 8086 ts_tablet_manager.cc:1397] T 7bdddf684a8c435290333be03b313b8a P 7a1c2468f20444faa40177d645c9a1ca: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260808 22:26:39.752499 7132 catalog_manager.cc:4841] TS 570f0cfa261644d98997a916932ad7df (127.5.11.194:39773): tablet ca7986efe42c4fa79956e4fb00f9fc4a (table test-table [id=3f040f8acd544b2fb2d783e7c5e1d36b]) successfully deleted
I20260808 22:26:39.752905 8086 raft_consensus.cc:348] T 7bdddf684a8c435290333be03b313b8a P 7a1c2468f20444faa40177d645c9a1ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } }
I20260808 22:26:39.753006 8086 raft_consensus.cc:374] T 7bdddf684a8c435290333be03b313b8a P 7a1c2468f20444faa40177d645c9a1ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:39.753036 8086 raft_consensus.cc:729] T 7bdddf684a8c435290333be03b313b8a P 7a1c2468f20444faa40177d645c9a1ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a1c2468f20444faa40177d645c9a1ca, State: Initialized, Role: FOLLOWER
I20260808 22:26:39.753153 8086 consensus_queue.cc:260] T 7bdddf684a8c435290333be03b313b8a P 7a1c2468f20444faa40177d645c9a1ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } }
I20260808 22:26:39.753400 8086 ts_tablet_manager.cc:1428] T 7bdddf684a8c435290333be03b313b8a P 7a1c2468f20444faa40177d645c9a1ca: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 22:26:39.753778 8085 tablet_bootstrap.cc:492] T 7bdddf684a8c435290333be03b313b8a P 34054211be954fcc9530b6b4ebc0ac4e: Bootstrap starting.
I20260808 22:26:39.754670 8085 tablet_bootstrap.cc:654] T 7bdddf684a8c435290333be03b313b8a P 34054211be954fcc9530b6b4ebc0ac4e: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:39.754881 8084 ts_tablet_manager.cc:1428] T 7bdddf684a8c435290333be03b313b8a P 570f0cfa261644d98997a916932ad7df: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260808 22:26:39.755925 8085 tablet_bootstrap.cc:492] T 7bdddf684a8c435290333be03b313b8a P 34054211be954fcc9530b6b4ebc0ac4e: No bootstrap required, opened a new log
I20260808 22:26:39.756019 8085 ts_tablet_manager.cc:1397] T 7bdddf684a8c435290333be03b313b8a P 34054211be954fcc9530b6b4ebc0ac4e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 22:26:39.756608 8085 raft_consensus.cc:348] T 7bdddf684a8c435290333be03b313b8a P 34054211be954fcc9530b6b4ebc0ac4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } }
I20260808 22:26:39.756698 8085 raft_consensus.cc:374] T 7bdddf684a8c435290333be03b313b8a P 34054211be954fcc9530b6b4ebc0ac4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:39.756721 8085 raft_consensus.cc:729] T 7bdddf684a8c435290333be03b313b8a P 34054211be954fcc9530b6b4ebc0ac4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34054211be954fcc9530b6b4ebc0ac4e, State: Initialized, Role: FOLLOWER
I20260808 22:26:39.756856 8085 consensus_queue.cc:260] T 7bdddf684a8c435290333be03b313b8a P 34054211be954fcc9530b6b4ebc0ac4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } }
I20260808 22:26:39.757138 8085 ts_tablet_manager.cc:1428] T 7bdddf684a8c435290333be03b313b8a P 34054211be954fcc9530b6b4ebc0ac4e: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
W20260808 22:26:39.757514 7530 tablet.cc:2369] T 7bdddf684a8c435290333be03b313b8a P 34054211be954fcc9530b6b4ebc0ac4e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 22:26:39.973801 7780 tablet.cc:2369] T 7bdddf684a8c435290333be03b313b8a P 570f0cfa261644d98997a916932ad7df: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:26:39.983520 8092 raft_consensus.cc:482] T 7bdddf684a8c435290333be03b313b8a P 34054211be954fcc9530b6b4ebc0ac4e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:26:39.983670 8092 raft_consensus.cc:504] T 7bdddf684a8c435290333be03b313b8a P 34054211be954fcc9530b6b4ebc0ac4e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } }
I20260808 22:26:39.984834 8092 leader_election.cc:288] T 7bdddf684a8c435290333be03b313b8a P 34054211be954fcc9530b6b4ebc0ac4e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 570f0cfa261644d98997a916932ad7df (127.5.11.194:39773), 7a1c2468f20444faa40177d645c9a1ca (127.5.11.195:38875)
I20260808 22:26:39.989133 8139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:39.985114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:38875 (local address 127.5.11.193:42561)
0808 22:26:39.985554 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:39.985567 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:39.985680 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:39.985972 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:39.985974 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:39.985989 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:26:39.986127 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:39.986133 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.986895 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.986898 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:39.987716 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:39.987725 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.988670 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.988674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:39.988790 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:39.988814 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 22:26:39.988880 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 22:26:39.988932 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"mutex_wait_us":27,"thread_start_us":118,"threads_started":1}
I20260808 22:26:39.989153 8078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:39.985210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:42561 (local address 127.5.11.195:38875)
0808 22:26:39.985353 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:39.985358 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:39.985670 (+ 312us) server_negotiation.cc:407] Connection header received
0808 22:26:39.985806 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:39.985809 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:39.985855 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:39.985936 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:26:39.986244 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.986762 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.987916 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.988520 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.988640 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:39.988694 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:39.988928 (+ 234us) server_negotiation.cc:299] Negotiation successful
0808 22:26:39.989021 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 22:26:39.989234 8080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:39.985249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:55337 (local address 127.5.11.194:39773)
0808 22:26:39.985388 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:39.985393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:39.985525 (+ 132us) server_negotiation.cc:407] Connection header received
0808 22:26:39.985655 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:39.985658 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:39.985709 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:39.985794 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:26:39.986125 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.986660 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.987944 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.988528 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.988640 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:39.988694 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:39.989031 (+ 337us) server_negotiation.cc:299] Negotiation successful
0808 22:26:39.989108 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 22:26:39.989192 8137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:39.985092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:39773 (local address 127.5.11.193:55337)
0808 22:26:39.985433 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:39.985445 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:39.985548 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:39.985813 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:39.985815 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:39.985834 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:26:39.985984 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:39.986004 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.986783 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.986787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:39.987775 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:39.987787 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:39.988757 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:39.988760 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:39.988855 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:39.988906 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 22:26:39.988987 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 22:26:39.989047 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"mutex_wait_us":5,"thread_start_us":83,"threads_started":1}
I20260808 22:26:39.989611 7954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bdddf684a8c435290333be03b313b8a" candidate_uuid: "34054211be954fcc9530b6b4ebc0ac4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a1c2468f20444faa40177d645c9a1ca" is_pre_election: true
I20260808 22:26:39.989787 7954 raft_consensus.cc:2454] T 7bdddf684a8c435290333be03b313b8a P 7a1c2468f20444faa40177d645c9a1ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34054211be954fcc9530b6b4ebc0ac4e in term 0.
I20260808 22:26:39.989934 7696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bdddf684a8c435290333be03b313b8a" candidate_uuid: "34054211be954fcc9530b6b4ebc0ac4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "570f0cfa261644d98997a916932ad7df" is_pre_election: true
I20260808 22:26:39.990207 7696 raft_consensus.cc:2454] T 7bdddf684a8c435290333be03b313b8a P 570f0cfa261644d98997a916932ad7df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34054211be954fcc9530b6b4ebc0ac4e in term 0.
I20260808 22:26:39.990201 7316 leader_election.cc:302] T 7bdddf684a8c435290333be03b313b8a P 34054211be954fcc9530b6b4ebc0ac4e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34054211be954fcc9530b6b4ebc0ac4e, 7a1c2468f20444faa40177d645c9a1ca; no voters:
I20260808 22:26:39.990482 8092 raft_consensus.cc:2781] T 7bdddf684a8c435290333be03b313b8a P 34054211be954fcc9530b6b4ebc0ac4e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:26:39.990552 8092 raft_consensus.cc:482] T 7bdddf684a8c435290333be03b313b8a P 34054211be954fcc9530b6b4ebc0ac4e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:26:39.990602 8092 raft_consensus.cc:3037] T 7bdddf684a8c435290333be03b313b8a P 34054211be954fcc9530b6b4ebc0ac4e [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:39.991822 8092 raft_consensus.cc:504] T 7bdddf684a8c435290333be03b313b8a P 34054211be954fcc9530b6b4ebc0ac4e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } }
I20260808 22:26:39.992175 8092 leader_election.cc:288] T 7bdddf684a8c435290333be03b313b8a P 34054211be954fcc9530b6b4ebc0ac4e [CANDIDATE]: Term 1 election: Requested vote from peers 570f0cfa261644d98997a916932ad7df (127.5.11.194:39773), 7a1c2468f20444faa40177d645c9a1ca (127.5.11.195:38875)
I20260808 22:26:39.992569 7696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bdddf684a8c435290333be03b313b8a" candidate_uuid: "34054211be954fcc9530b6b4ebc0ac4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "570f0cfa261644d98997a916932ad7df"
I20260808 22:26:39.992615 7954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7bdddf684a8c435290333be03b313b8a" candidate_uuid: "34054211be954fcc9530b6b4ebc0ac4e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7a1c2468f20444faa40177d645c9a1ca"
I20260808 22:26:39.992694 7696 raft_consensus.cc:3037] T 7bdddf684a8c435290333be03b313b8a P 570f0cfa261644d98997a916932ad7df [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:39.992712 7954 raft_consensus.cc:3037] T 7bdddf684a8c435290333be03b313b8a P 7a1c2468f20444faa40177d645c9a1ca [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:39.993868 7954 raft_consensus.cc:2454] T 7bdddf684a8c435290333be03b313b8a P 7a1c2468f20444faa40177d645c9a1ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34054211be954fcc9530b6b4ebc0ac4e in term 1.
I20260808 22:26:39.993886 7696 raft_consensus.cc:2454] T 7bdddf684a8c435290333be03b313b8a P 570f0cfa261644d98997a916932ad7df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34054211be954fcc9530b6b4ebc0ac4e in term 1.
I20260808 22:26:39.994138 7316 leader_election.cc:302] T 7bdddf684a8c435290333be03b313b8a P 34054211be954fcc9530b6b4ebc0ac4e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34054211be954fcc9530b6b4ebc0ac4e, 7a1c2468f20444faa40177d645c9a1ca; no voters:
I20260808 22:26:39.994315 8092 raft_consensus.cc:2781] T 7bdddf684a8c435290333be03b313b8a P 34054211be954fcc9530b6b4ebc0ac4e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:26:39.994555 8092 raft_consensus.cc:686] T 7bdddf684a8c435290333be03b313b8a P 34054211be954fcc9530b6b4ebc0ac4e [term 1 LEADER]: Becoming Leader. State: Replica: 34054211be954fcc9530b6b4ebc0ac4e, State: Running, Role: LEADER
I20260808 22:26:39.994840 8092 consensus_queue.cc:237] T 7bdddf684a8c435290333be03b313b8a P 34054211be954fcc9530b6b4ebc0ac4e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } }
I20260808 22:26:39.997290 7157 catalog_manager.cc:5495] T 7bdddf684a8c435290333be03b313b8a P 34054211be954fcc9530b6b4ebc0ac4e reported cstate change: term changed from 0 to 1, leader changed from <none> to 34054211be954fcc9530b6b4ebc0ac4e (127.5.11.193). New cstate: current_term: 1 leader_uuid: "34054211be954fcc9530b6b4ebc0ac4e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } health_report { overall_health: UNKNOWN } } }
Aug 08 22:26:40 dist-test-slave-7f01 krb5kdc[7082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227998, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:26:40.078517 8151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:40.069463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39505 (local address 127.0.0.1:56746)
0808 22:26:40.069845 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:40.069862 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:40.070012 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:40.070272 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:40.070275 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:40.070795 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:40.070890 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:40.070897 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:40.071683 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:40.071689 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:40.072315 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:40.072325 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:40.072782 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:40.072785 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:40.072858 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:40.073525 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:26:40.073537 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:40.074981 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:40.076857 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:40.076867 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:40.076873 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:40.077126 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:40.077435 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:40.077440 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:40.077442 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:40.077527 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:40.077913 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:40.077920 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:40.078034 (+ 114us) client_negotiation.cc:769] Sending connection context
0808 22:26:40.078153 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 22:26:40.078311 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":108,"threads_started":1}
I20260808 22:26:40.078518 8154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:40.069566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56746 (local address 127.5.11.254:39505)
0808 22:26:40.069848 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:40.069862 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:40.069948 (+ 86us) server_negotiation.cc:407] Connection header received
0808 22:26:40.070110 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:40.070113 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:40.070159 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:40.070255 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:40.071011 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:40.071572 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:40.072439 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:40.072669 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:40.072782 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:40.075119 (+ 2337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:40.075122 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:40.075127 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:40.075152 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:40.076717 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:40.077238 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:40.077243 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:40.077245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:40.077327 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:40.077616 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:40.077621 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:40.077623 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:40.077803 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:40.077913 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:40.078200 (+ 287us) server_negotiation.cc:299] Negotiation successful
0808 22:26:40.078365 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":175,"thread_start_us":93,"threads_started":1}
I20260808 22:26:40.082842 7157 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:56746:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 22:26:40.083573 7157 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:26:40.088495 7414 tablet_service.cc:1467] Processing CreateTablet for tablet 97d1cc9a1a4a4f46ac02228bfde3861a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5d974ac155b641f18a94ee9b57ccc502]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 22:26:40.088630 7658 tablet_service.cc:1467] Processing CreateTablet for tablet 97d1cc9a1a4a4f46ac02228bfde3861a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5d974ac155b641f18a94ee9b57ccc502]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 22:26:40.088711 7915 tablet_service.cc:1467] Processing CreateTablet for tablet 97d1cc9a1a4a4f46ac02228bfde3861a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5d974ac155b641f18a94ee9b57ccc502]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 22:26:40.088929 7414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97d1cc9a1a4a4f46ac02228bfde3861a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:40.089038 7915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97d1cc9a1a4a4f46ac02228bfde3861a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:40.089040 7658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97d1cc9a1a4a4f46ac02228bfde3861a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:40.091480 8085 tablet_bootstrap.cc:492] T 97d1cc9a1a4a4f46ac02228bfde3861a P 34054211be954fcc9530b6b4ebc0ac4e: Bootstrap starting.
I20260808 22:26:40.091812 8084 tablet_bootstrap.cc:492] T 97d1cc9a1a4a4f46ac02228bfde3861a P 570f0cfa261644d98997a916932ad7df: Bootstrap starting.
I20260808 22:26:40.092439 8085 tablet_bootstrap.cc:654] T 97d1cc9a1a4a4f46ac02228bfde3861a P 34054211be954fcc9530b6b4ebc0ac4e: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:40.092648 8084 tablet_bootstrap.cc:654] T 97d1cc9a1a4a4f46ac02228bfde3861a P 570f0cfa261644d98997a916932ad7df: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:40.093173 8086 tablet_bootstrap.cc:492] T 97d1cc9a1a4a4f46ac02228bfde3861a P 7a1c2468f20444faa40177d645c9a1ca: Bootstrap starting.
I20260808 22:26:40.093923 8085 tablet_bootstrap.cc:492] T 97d1cc9a1a4a4f46ac02228bfde3861a P 34054211be954fcc9530b6b4ebc0ac4e: No bootstrap required, opened a new log
I20260808 22:26:40.094009 8085 ts_tablet_manager.cc:1397] T 97d1cc9a1a4a4f46ac02228bfde3861a P 34054211be954fcc9530b6b4ebc0ac4e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:26:40.094203 8086 tablet_bootstrap.cc:654] T 97d1cc9a1a4a4f46ac02228bfde3861a P 7a1c2468f20444faa40177d645c9a1ca: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:40.094198 8084 tablet_bootstrap.cc:492] T 97d1cc9a1a4a4f46ac02228bfde3861a P 570f0cfa261644d98997a916932ad7df: No bootstrap required, opened a new log
I20260808 22:26:40.094285 8084 ts_tablet_manager.cc:1397] T 97d1cc9a1a4a4f46ac02228bfde3861a P 570f0cfa261644d98997a916932ad7df: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:26:40.094513 8085 raft_consensus.cc:348] T 97d1cc9a1a4a4f46ac02228bfde3861a P 34054211be954fcc9530b6b4ebc0ac4e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } }
I20260808 22:26:40.094627 8085 raft_consensus.cc:374] T 97d1cc9a1a4a4f46ac02228bfde3861a P 34054211be954fcc9530b6b4ebc0ac4e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:40.094671 8085 raft_consensus.cc:729] T 97d1cc9a1a4a4f46ac02228bfde3861a P 34054211be954fcc9530b6b4ebc0ac4e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34054211be954fcc9530b6b4ebc0ac4e, State: Initialized, Role: FOLLOWER
I20260808 22:26:40.094625 8084 raft_consensus.cc:348] T 97d1cc9a1a4a4f46ac02228bfde3861a P 570f0cfa261644d98997a916932ad7df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } }
I20260808 22:26:40.094745 8084 raft_consensus.cc:374] T 97d1cc9a1a4a4f46ac02228bfde3861a P 570f0cfa261644d98997a916932ad7df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:40.094779 8084 raft_consensus.cc:729] T 97d1cc9a1a4a4f46ac02228bfde3861a P 570f0cfa261644d98997a916932ad7df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 570f0cfa261644d98997a916932ad7df, State: Initialized, Role: FOLLOWER
I20260808 22:26:40.094795 8085 consensus_queue.cc:260] T 97d1cc9a1a4a4f46ac02228bfde3861a P 34054211be954fcc9530b6b4ebc0ac4e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } }
I20260808 22:26:40.094903 8084 consensus_queue.cc:260] T 97d1cc9a1a4a4f46ac02228bfde3861a P 570f0cfa261644d98997a916932ad7df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } }
I20260808 22:26:40.095199 8084 ts_tablet_manager.cc:1428] T 97d1cc9a1a4a4f46ac02228bfde3861a P 570f0cfa261644d98997a916932ad7df: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:26:40.095158 8092 tablet_replica.cc:439] T 97d1cc9a1a4a4f46ac02228bfde3861a P 34054211be954fcc9530b6b4ebc0ac4e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } } }
I20260808 22:26:40.095301 8092 tablet_replica.cc:442] T 97d1cc9a1a4a4f46ac02228bfde3861a P 34054211be954fcc9530b6b4ebc0ac4e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:26:40.095435 8101 tablet_replica.cc:439] T 97d1cc9a1a4a4f46ac02228bfde3861a P 570f0cfa261644d98997a916932ad7df: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } } }
I20260808 22:26:40.095561 8101 tablet_replica.cc:442] T 97d1cc9a1a4a4f46ac02228bfde3861a P 570f0cfa261644d98997a916932ad7df: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:26:40.095566 8086 tablet_bootstrap.cc:492] T 97d1cc9a1a4a4f46ac02228bfde3861a P 7a1c2468f20444faa40177d645c9a1ca: No bootstrap required, opened a new log
I20260808 22:26:40.095640 8086 ts_tablet_manager.cc:1397] T 97d1cc9a1a4a4f46ac02228bfde3861a P 7a1c2468f20444faa40177d645c9a1ca: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:26:40.095964 8085 ts_tablet_manager.cc:1428] T 97d1cc9a1a4a4f46ac02228bfde3861a P 34054211be954fcc9530b6b4ebc0ac4e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260808 22:26:40.096108 8086 raft_consensus.cc:348] T 97d1cc9a1a4a4f46ac02228bfde3861a P 7a1c2468f20444faa40177d645c9a1ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } }
I20260808 22:26:40.096237 8086 raft_consensus.cc:374] T 97d1cc9a1a4a4f46ac02228bfde3861a P 7a1c2468f20444faa40177d645c9a1ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:40.096278 8086 raft_consensus.cc:729] T 97d1cc9a1a4a4f46ac02228bfde3861a P 7a1c2468f20444faa40177d645c9a1ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a1c2468f20444faa40177d645c9a1ca, State: Initialized, Role: FOLLOWER
I20260808 22:26:40.096426 8086 consensus_queue.cc:260] T 97d1cc9a1a4a4f46ac02228bfde3861a P 7a1c2468f20444faa40177d645c9a1ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } }
I20260808 22:26:40.096697 8086 ts_tablet_manager.cc:1428] T 97d1cc9a1a4a4f46ac02228bfde3861a P 7a1c2468f20444faa40177d645c9a1ca: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 22:26:40.096657 8094 tablet_replica.cc:439] T 97d1cc9a1a4a4f46ac02228bfde3861a P 7a1c2468f20444faa40177d645c9a1ca: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } } }
I20260808 22:26:40.096784 8094 tablet_replica.cc:442] T 97d1cc9a1a4a4f46ac02228bfde3861a P 7a1c2468f20444faa40177d645c9a1ca: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:26:40.137924 8094 raft_consensus.cc:482] T 97d1cc9a1a4a4f46ac02228bfde3861a P 7a1c2468f20444faa40177d645c9a1ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:26:40.138048 8094 raft_consensus.cc:504] T 97d1cc9a1a4a4f46ac02228bfde3861a P 7a1c2468f20444faa40177d645c9a1ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } }
I20260808 22:26:40.139238 8094 leader_election.cc:288] T 97d1cc9a1a4a4f46ac02228bfde3861a P 7a1c2468f20444faa40177d645c9a1ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 34054211be954fcc9530b6b4ebc0ac4e (127.5.11.193:44039), 570f0cfa261644d98997a916932ad7df (127.5.11.194:39773)
I20260808 22:26:40.144484 8080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:40.139596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:60885 (local address 127.5.11.194:39773)
0808 22:26:40.139772 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:40.139779 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:40.139938 (+ 159us) server_negotiation.cc:407] Connection header received
0808 22:26:40.140027 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:40.140030 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:40.140083 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:40.140165 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:26:40.140656 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:40.141440 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:40.143201 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:40.143826 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:40.143950 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:40.144004 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:40.144283 (+ 279us) server_negotiation.cc:299] Negotiation successful
0808 22:26:40.144353 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 22:26:40.144483 8077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:40.139552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:41259 (local address 127.5.11.193:44039)
0808 22:26:40.139675 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:40.139680 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:40.139856 (+ 176us) server_negotiation.cc:407] Connection header received
0808 22:26:40.140028 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:40.140031 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:40.140079 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:40.140165 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:26:40.140656 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:40.141440 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:40.143201 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:40.143826 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:40.143950 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:40.144017 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:40.144283 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 22:26:40.144353 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 22:26:40.144560 8159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:40.139395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:44039 (local address 127.5.11.195:41259)
0808 22:26:40.139754 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:40.139773 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:40.139896 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:40.140193 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:40.140196 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:40.140241 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:26:40.140530 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:40.140537 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:40.141556 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:40.141560 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:40.143059 (+ 1499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:40.143075 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:40.143955 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:40.143959 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:40.144121 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:40.144167 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 22:26:40.144249 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 22:26:40.144369 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":257,"spinlock_wait_cycles":9856,"thread_start_us":89,"threads_started":1}
I20260808 22:26:40.145009 7449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97d1cc9a1a4a4f46ac02228bfde3861a" candidate_uuid: "7a1c2468f20444faa40177d645c9a1ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34054211be954fcc9530b6b4ebc0ac4e" is_pre_election: true
I20260808 22:26:40.145063 8160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:40.139474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:39773 (local address 127.5.11.195:60885)
0808 22:26:40.139852 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:40.139865 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:40.139948 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:40.140193 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:40.140196 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:40.140239 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:26:40.140530 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:40.140537 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:40.141556 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:40.141560 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:40.143071 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:40.143077 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:40.143955 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:40.143959 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:40.144125 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:40.144172 (+ 47us) client_negotiation.cc:769] Sending connection context
0808 22:26:40.144249 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 22:26:40.144373 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":274,"spinlock_wait_cycles":33664,"thread_start_us":101,"threads_started":1}
I20260808 22:26:40.145138 7449 raft_consensus.cc:2454] T 97d1cc9a1a4a4f46ac02228bfde3861a P 34054211be954fcc9530b6b4ebc0ac4e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a1c2468f20444faa40177d645c9a1ca in term 0.
I20260808 22:26:40.145539 7834 leader_election.cc:302] T 97d1cc9a1a4a4f46ac02228bfde3861a P 7a1c2468f20444faa40177d645c9a1ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34054211be954fcc9530b6b4ebc0ac4e, 7a1c2468f20444faa40177d645c9a1ca; no voters:
I20260808 22:26:40.145691 7696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97d1cc9a1a4a4f46ac02228bfde3861a" candidate_uuid: "7a1c2468f20444faa40177d645c9a1ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "570f0cfa261644d98997a916932ad7df" is_pre_election: true
I20260808 22:26:40.145793 8094 raft_consensus.cc:2781] T 97d1cc9a1a4a4f46ac02228bfde3861a P 7a1c2468f20444faa40177d645c9a1ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:26:40.145838 7696 raft_consensus.cc:2454] T 97d1cc9a1a4a4f46ac02228bfde3861a P 570f0cfa261644d98997a916932ad7df [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a1c2468f20444faa40177d645c9a1ca in term 0.
I20260808 22:26:40.145861 8094 raft_consensus.cc:482] T 97d1cc9a1a4a4f46ac02228bfde3861a P 7a1c2468f20444faa40177d645c9a1ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:26:40.145905 8094 raft_consensus.cc:3037] T 97d1cc9a1a4a4f46ac02228bfde3861a P 7a1c2468f20444faa40177d645c9a1ca [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:40.146895 8094 raft_consensus.cc:504] T 97d1cc9a1a4a4f46ac02228bfde3861a P 7a1c2468f20444faa40177d645c9a1ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } }
I20260808 22:26:40.147271 8094 leader_election.cc:288] T 97d1cc9a1a4a4f46ac02228bfde3861a P 7a1c2468f20444faa40177d645c9a1ca [CANDIDATE]: Term 1 election: Requested vote from peers 34054211be954fcc9530b6b4ebc0ac4e (127.5.11.193:44039), 570f0cfa261644d98997a916932ad7df (127.5.11.194:39773)
I20260808 22:26:40.147608 7696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97d1cc9a1a4a4f46ac02228bfde3861a" candidate_uuid: "7a1c2468f20444faa40177d645c9a1ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "570f0cfa261644d98997a916932ad7df"
I20260808 22:26:40.147608 7449 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97d1cc9a1a4a4f46ac02228bfde3861a" candidate_uuid: "7a1c2468f20444faa40177d645c9a1ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34054211be954fcc9530b6b4ebc0ac4e"
I20260808 22:26:40.147723 7449 raft_consensus.cc:3037] T 97d1cc9a1a4a4f46ac02228bfde3861a P 34054211be954fcc9530b6b4ebc0ac4e [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:40.147723 7696 raft_consensus.cc:3037] T 97d1cc9a1a4a4f46ac02228bfde3861a P 570f0cfa261644d98997a916932ad7df [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:40.148859 7696 raft_consensus.cc:2454] T 97d1cc9a1a4a4f46ac02228bfde3861a P 570f0cfa261644d98997a916932ad7df [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a1c2468f20444faa40177d645c9a1ca in term 1.
I20260808 22:26:40.148864 7449 raft_consensus.cc:2454] T 97d1cc9a1a4a4f46ac02228bfde3861a P 34054211be954fcc9530b6b4ebc0ac4e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a1c2468f20444faa40177d645c9a1ca in term 1.
I20260808 22:26:40.149124 7834 leader_election.cc:302] T 97d1cc9a1a4a4f46ac02228bfde3861a P 7a1c2468f20444faa40177d645c9a1ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34054211be954fcc9530b6b4ebc0ac4e, 7a1c2468f20444faa40177d645c9a1ca; no voters:
I20260808 22:26:40.149304 8094 raft_consensus.cc:2781] T 97d1cc9a1a4a4f46ac02228bfde3861a P 7a1c2468f20444faa40177d645c9a1ca [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:26:40.149501 8094 raft_consensus.cc:686] T 97d1cc9a1a4a4f46ac02228bfde3861a P 7a1c2468f20444faa40177d645c9a1ca [term 1 LEADER]: Becoming Leader. State: Replica: 7a1c2468f20444faa40177d645c9a1ca, State: Running, Role: LEADER
I20260808 22:26:40.149740 8094 consensus_queue.cc:237] T 97d1cc9a1a4a4f46ac02228bfde3861a P 7a1c2468f20444faa40177d645c9a1ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } }
I20260808 22:26:40.151551 8161 tablet_replica.cc:439] T 97d1cc9a1a4a4f46ac02228bfde3861a P 7a1c2468f20444faa40177d645c9a1ca: TxnStatusTablet state changed. Reason: New leader 7a1c2468f20444faa40177d645c9a1ca. Latest consensus state: current_term: 1 leader_uuid: "7a1c2468f20444faa40177d645c9a1ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } } }
I20260808 22:26:40.151690 8161 tablet_replica.cc:442] T 97d1cc9a1a4a4f46ac02228bfde3861a P 7a1c2468f20444faa40177d645c9a1ca: This TxnStatusTablet replica's current role is: LEADER
I20260808 22:26:40.152047 8164 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 22:26:40.152243 8164 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 22:26:40.152388 7157 catalog_manager.cc:5495] T 97d1cc9a1a4a4f46ac02228bfde3861a P 7a1c2468f20444faa40177d645c9a1ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a1c2468f20444faa40177d645c9a1ca (127.5.11.195). New cstate: current_term: 1 leader_uuid: "7a1c2468f20444faa40177d645c9a1ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } health_report { overall_health: HEALTHY } } }
I20260808 22:26:40.164664 8078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:40.161119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48090 (local address 127.5.11.195:38875)
0808 22:26:40.161222 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:40.161227 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:40.161265 (+ 38us) server_negotiation.cc:407] Connection header received
0808 22:26:40.161471 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:40.161476 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:40.161529 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:40.161635 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:26:40.161906 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:40.162387 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:40.163241 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:40.163463 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:40.163574 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:40.163922 (+ 348us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:26:40.164285 (+ 363us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:26:40.164417 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:40.164491 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 22:26:40.164538 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 22:26:40.164681 8151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:40.161045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:38875 (local address 127.0.0.1:48090)
0808 22:26:40.161202 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:40.161218 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:40.161335 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:40.161663 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:40.161666 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:40.161676 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:26:40.161779 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:40.161784 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:40.162501 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:40.162505 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:40.163117 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:40.163126 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:40.163576 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:40.163580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:40.163684 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:40.163726 (+ 42us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:26:40.164394 (+ 668us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:26:40.164412 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 22:26:40.164483 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 22:26:40.164568 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 22:26:40.167430 7449 raft_consensus.cc:1261] T 97d1cc9a1a4a4f46ac02228bfde3861a P 34054211be954fcc9530b6b4ebc0ac4e [term 1 FOLLOWER]: Refusing update from remote peer 7a1c2468f20444faa40177d645c9a1ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:26:40.167610 7696 raft_consensus.cc:1261] T 97d1cc9a1a4a4f46ac02228bfde3861a P 570f0cfa261644d98997a916932ad7df [term 1 FOLLOWER]: Refusing update from remote peer 7a1c2468f20444faa40177d645c9a1ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:26:40.168267 8161 consensus_queue.cc:1035] T 97d1cc9a1a4a4f46ac02228bfde3861a P 7a1c2468f20444faa40177d645c9a1ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:26:40.168462 8094 consensus_queue.cc:1035] T 97d1cc9a1a4a4f46ac02228bfde3861a P 7a1c2468f20444faa40177d645c9a1ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:26:40.170027 8101 tablet_replica.cc:439] T 97d1cc9a1a4a4f46ac02228bfde3861a P 570f0cfa261644d98997a916932ad7df: TxnStatusTablet state changed. Reason: New leader 7a1c2468f20444faa40177d645c9a1ca. Latest consensus state: current_term: 1 leader_uuid: "7a1c2468f20444faa40177d645c9a1ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } } }
I20260808 22:26:40.170137 8101 tablet_replica.cc:442] T 97d1cc9a1a4a4f46ac02228bfde3861a P 570f0cfa261644d98997a916932ad7df: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:26:40.170099 8092 tablet_replica.cc:439] T 97d1cc9a1a4a4f46ac02228bfde3861a P 34054211be954fcc9530b6b4ebc0ac4e: TxnStatusTablet state changed. Reason: New leader 7a1c2468f20444faa40177d645c9a1ca. Latest consensus state: current_term: 1 leader_uuid: "7a1c2468f20444faa40177d645c9a1ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } } }
I20260808 22:26:40.170233 8092 tablet_replica.cc:442] T 97d1cc9a1a4a4f46ac02228bfde3861a P 34054211be954fcc9530b6b4ebc0ac4e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:26:40.172570 8094 tablet_replica.cc:439] T 97d1cc9a1a4a4f46ac02228bfde3861a P 7a1c2468f20444faa40177d645c9a1ca: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7a1c2468f20444faa40177d645c9a1ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } } }
I20260808 22:26:40.172681 8094 tablet_replica.cc:442] T 97d1cc9a1a4a4f46ac02228bfde3861a P 7a1c2468f20444faa40177d645c9a1ca: This TxnStatusTablet replica's current role is: LEADER
I20260808 22:26:40.173609 8161 tablet_replica.cc:439] T 97d1cc9a1a4a4f46ac02228bfde3861a P 7a1c2468f20444faa40177d645c9a1ca: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7a1c2468f20444faa40177d645c9a1ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } } }
I20260808 22:26:40.173717 8161 tablet_replica.cc:442] T 97d1cc9a1a4a4f46ac02228bfde3861a P 7a1c2468f20444faa40177d645c9a1ca: This TxnStatusTablet replica's current role is: LEADER
I20260808 22:26:40.174566 8101 tablet_replica.cc:439] T 97d1cc9a1a4a4f46ac02228bfde3861a P 570f0cfa261644d98997a916932ad7df: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7a1c2468f20444faa40177d645c9a1ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } } }
I20260808 22:26:40.174670 8101 tablet_replica.cc:442] T 97d1cc9a1a4a4f46ac02228bfde3861a P 570f0cfa261644d98997a916932ad7df: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:26:40.174875 8092 tablet_replica.cc:439] T 97d1cc9a1a4a4f46ac02228bfde3861a P 34054211be954fcc9530b6b4ebc0ac4e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7a1c2468f20444faa40177d645c9a1ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "34054211be954fcc9530b6b4ebc0ac4e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44039 } } peers { permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 } } peers { permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 } } }
I20260808 22:26:40.174979 8092 tablet_replica.cc:442] T 97d1cc9a1a4a4f46ac02228bfde3861a P 34054211be954fcc9530b6b4ebc0ac4e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:26:40.182782 8077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:40.178741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60486 (local address 127.5.11.193:44039)
0808 22:26:40.178844 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:40.178849 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:40.178861 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:26:40.179030 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:40.179033 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:40.179091 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:40.179196 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:26:40.179394 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:40.180181 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:40.181073 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:40.181418 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:40.181527 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:40.181832 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:26:40.182356 (+ 524us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:26:40.182446 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:40.182585 (+ 139us) server_negotiation.cc:299] Negotiation successful
0808 22:26:40.182633 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 22:26:40.193892 8078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:40.189971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:39147 (local address 127.5.11.195:38875)
0808 22:26:40.190084 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:40.190088 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:40.190262 (+ 174us) server_negotiation.cc:407] Connection header received
0808 22:26:40.190344 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:40.190347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:40.190395 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:40.190475 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:26:40.191018 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:40.191823 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:40.192661 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:40.193034 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:40.193132 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:40.193443 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:26:40.193549 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:26:40.193619 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:40.193712 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 22:26:40.193765 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 22:26:40.194092 8182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:40.189856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:38875 (local address 127.5.11.193:39147)
0808 22:26:40.190174 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:40.190187 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:40.190272 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:40.190493 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:40.190495 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:40.190501 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:26:40.190905 (+ 404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:40.190911 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:40.191924 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:40.191927 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:40.192549 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:40.192557 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:40.193168 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:40.193171 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:40.193255 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:40.193301 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:26:40.193621 (+ 320us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:26:40.193627 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 22:26:40.193679 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 22:26:40.193741 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":121,"threads_started":1}
I20260808 22:26:40.202657 7954 raft_consensus.cc:1261] T 7bdddf684a8c435290333be03b313b8a P 7a1c2468f20444faa40177d645c9a1ca [term 1 FOLLOWER]: Refusing update from remote peer 34054211be954fcc9530b6b4ebc0ac4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:26:40.202677 7696 raft_consensus.cc:1261] T 7bdddf684a8c435290333be03b313b8a P 570f0cfa261644d98997a916932ad7df [term 1 FOLLOWER]: Refusing update from remote peer 34054211be954fcc9530b6b4ebc0ac4e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:26:40.203131 8092 consensus_queue.cc:1035] T 7bdddf684a8c435290333be03b313b8a P 34054211be954fcc9530b6b4ebc0ac4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "570f0cfa261644d98997a916932ad7df" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 39773 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:26:40.203337 8143 consensus_queue.cc:1035] T 7bdddf684a8c435290333be03b313b8a P 34054211be954fcc9530b6b4ebc0ac4e [LEADER]: Connected to new peer: Peer: permanent_uuid: "7a1c2468f20444faa40177d645c9a1ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:26:40.208266 8104 mvcc.cc:205] Tried to move back new op lower bound from 7316389888825397248 to 7316389887981789184. Current Snapshot: MvccSnapshot[applied={T|T < 7316389888825397248}]
I20260808 22:26:40.226893 8077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:40.222872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:50677 (local address 127.5.11.193:44039)
0808 22:26:40.222976 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:40.222981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:40.222995 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:26:40.223072 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:40.223075 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:40.223123 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:40.223201 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:26:40.223494 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:40.224285 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:40.225606 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:40.225940 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:40.226011 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:40.226412 (+ 401us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:26:40.226503 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:26:40.226547 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:40.226700 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 22:26:40.226749 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 22:26:40.227567 8193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:40.222557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:44039 (local address 127.5.11.195:50677)
0808 22:26:40.222877 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:40.222893 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:40.222987 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:40.223219 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:40.223222 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:40.223228 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:26:40.223370 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:40.223376 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:40.224419 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:40.224423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:40.225454 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:40.225467 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:40.226122 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:40.226125 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:40.226220 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:40.226269 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:26:40.226608 (+ 339us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:26:40.226614 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 22:26:40.227312 (+ 698us) client_negotiation.cc:240] Negotiation successful
0808 22:26:40.227394 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":100,"threads_started":1}
I20260808 22:26:40.258225 7157 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:26:40.258397 7157 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56736:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:26:40.260805 7157 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 576de68957c746e1b39b808dee28cac5: Sending DeleteTablet for 3 replicas of tablet 7bdddf684a8c435290333be03b313b8a
I20260808 22:26:40.261351 7414 tablet_service.cc:1514] Processing DeleteTablet for tablet 7bdddf684a8c435290333be03b313b8a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:26:40 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:60480
I20260808 22:26:40.261373 7658 tablet_service.cc:1514] Processing DeleteTablet for tablet 7bdddf684a8c435290333be03b313b8a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:26:40 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:41296
I20260808 22:26:40.261981 7915 tablet_service.cc:1514] Processing DeleteTablet for tablet 7bdddf684a8c435290333be03b313b8a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:26:40 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:48086
I20260808 22:26:40.263200 8198 tablet_replica.cc:330] T 7bdddf684a8c435290333be03b313b8a P 34054211be954fcc9530b6b4ebc0ac4e: stopping tablet replica
I20260808 22:26:40.263329 8198 raft_consensus.cc:2229] T 7bdddf684a8c435290333be03b313b8a P 34054211be954fcc9530b6b4ebc0ac4e [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:26:40.263620 8198 raft_consensus.cc:2258] T 7bdddf684a8c435290333be03b313b8a P 34054211be954fcc9530b6b4ebc0ac4e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:26:40.263888 8200 tablet_replica.cc:330] T 7bdddf684a8c435290333be03b313b8a P 570f0cfa261644d98997a916932ad7df: stopping tablet replica
I20260808 22:26:40.264011 8200 raft_consensus.cc:2229] T 7bdddf684a8c435290333be03b313b8a P 570f0cfa261644d98997a916932ad7df [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:26:40.264118 8200 raft_consensus.cc:2258] T 7bdddf684a8c435290333be03b313b8a P 570f0cfa261644d98997a916932ad7df [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:26:40.264803 8200 ts_tablet_manager.cc:1905] T 7bdddf684a8c435290333be03b313b8a P 570f0cfa261644d98997a916932ad7df: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:26:40.267891 8200 ts_tablet_manager.cc:1918] T 7bdddf684a8c435290333be03b313b8a P 570f0cfa261644d98997a916932ad7df: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:26:40.267975 8200 log.cc:1196] T 7bdddf684a8c435290333be03b313b8a P 570f0cfa261644d98997a916932ad7df: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-1/wal/wals/7bdddf684a8c435290333be03b313b8a
I20260808 22:26:40.268244 8200 ts_tablet_manager.cc:1939] T 7bdddf684a8c435290333be03b313b8a P 570f0cfa261644d98997a916932ad7df: Deleting consensus metadata
I20260808 22:26:40.268932 7132 catalog_manager.cc:4841] TS 570f0cfa261644d98997a916932ad7df (127.5.11.194:39773): tablet 7bdddf684a8c435290333be03b313b8a (table test-table [id=d42c567d68d54135a5b6aa31c94662a1]) successfully deleted
I20260808 22:26:40.269515 8198 ts_tablet_manager.cc:1905] T 7bdddf684a8c435290333be03b313b8a P 34054211be954fcc9530b6b4ebc0ac4e: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 08 22:26:40 dist-test-slave-7f01 krb5kdc[7082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786227999, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.11.193@KRBTEST.COM
I20260808 22:26:40.272291 8198 ts_tablet_manager.cc:1918] T 7bdddf684a8c435290333be03b313b8a P 34054211be954fcc9530b6b4ebc0ac4e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:26:40.272377 8198 log.cc:1196] T 7bdddf684a8c435290333be03b313b8a P 34054211be954fcc9530b6b4ebc0ac4e: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-0/wal/wals/7bdddf684a8c435290333be03b313b8a
I20260808 22:26:40.272636 8198 ts_tablet_manager.cc:1939] T 7bdddf684a8c435290333be03b313b8a P 34054211be954fcc9530b6b4ebc0ac4e: Deleting consensus metadata
I20260808 22:26:40.273268 7136 catalog_manager.cc:4841] TS 34054211be954fcc9530b6b4ebc0ac4e (127.5.11.193:44039): tablet 7bdddf684a8c435290333be03b313b8a (table test-table [id=d42c567d68d54135a5b6aa31c94662a1]) successfully deleted
I20260808 22:26:40.275921 8201 tablet_replica.cc:330] T 7bdddf684a8c435290333be03b313b8a P 7a1c2468f20444faa40177d645c9a1ca: stopping tablet replica
I20260808 22:26:40.276050 8201 raft_consensus.cc:2229] T 7bdddf684a8c435290333be03b313b8a P 7a1c2468f20444faa40177d645c9a1ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:26:40.276168 8201 raft_consensus.cc:2258] T 7bdddf684a8c435290333be03b313b8a P 7a1c2468f20444faa40177d645c9a1ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:26:40.276778 8201 ts_tablet_manager.cc:1905] T 7bdddf684a8c435290333be03b313b8a P 7a1c2468f20444faa40177d645c9a1ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:26:40.277519 8077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:40.263280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60496 (local address 127.5.11.193:44039)
0808 22:26:40.263408 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:40.263413 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:40.264158 (+ 745us) server_negotiation.cc:407] Connection header received
0808 22:26:40.264572 (+ 414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:40.264575 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:40.264618 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:40.264665 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:40.265663 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:40.266385 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:40.267685 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:40.268089 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:40.268166 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:40.273078 (+ 4912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:40.273088 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:40.273102 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:40.273128 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:40.275095 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:40.275605 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:40.275611 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:40.275613 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:40.275712 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:40.276177 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:40.276181 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:40.276183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:40.276505 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:40.276692 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:40.277210 (+ 518us) server_negotiation.cc:299] Negotiation successful
0808 22:26:40.277340 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260808 22:26:40.278558 7511 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:60496
I20260808 22:26:40.279397 8201 ts_tablet_manager.cc:1918] T 7bdddf684a8c435290333be03b313b8a P 7a1c2468f20444faa40177d645c9a1ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:26:40.279474 8201 log.cc:1196] T 7bdddf684a8c435290333be03b313b8a P 7a1c2468f20444faa40177d645c9a1ca: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786227991988852-5167-0/minicluster-data/ts-2/wal/wals/7bdddf684a8c435290333be03b313b8a
I20260808 22:26:40.279721 8201 ts_tablet_manager.cc:1939] T 7bdddf684a8c435290333be03b313b8a P 7a1c2468f20444faa40177d645c9a1ca: Deleting consensus metadata
I20260808 22:26:40.280416 7131 catalog_manager.cc:4841] TS 7a1c2468f20444faa40177d645c9a1ca (127.5.11.195:38875): tablet 7bdddf684a8c435290333be03b313b8a (table test-table [id=d42c567d68d54135a5b6aa31c94662a1]) successfully deleted
I20260808 22:26:40.290156 8154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:40.281156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56752 (local address 127.5.11.254:39505)
0808 22:26:40.281292 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:40.281297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:40.281368 (+ 71us) server_negotiation.cc:407] Connection header received
0808 22:26:40.281512 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:40.281515 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:40.281568 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:40.281672 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:40.282524 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:40.283357 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:40.284520 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:40.284888 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:40.284991 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:40.286567 (+ 1576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:40.286572 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:40.286576 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:40.286603 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:40.288355 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:40.288849 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:40.288853 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:40.288855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:40.288949 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:40.289254 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:40.289258 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:40.289260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:40.289468 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:40.289604 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:40.289864 (+ 260us) server_negotiation.cc:299] Negotiation successful
0808 22:26:40.289995 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260808 22:26:40.290489 7157 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56752
I20260808 22:26:40.292171 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 7289
I20260808 22:26:40.303934 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 7543
I20260808 22:26:40.322383 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 7800
I20260808 22:26:40.336313 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 7103
2026-08-08T22:26:40Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3208 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:26:40 dist-test-slave-7f01 krb5kdc[8212](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:26:40 dist-test-slave-7f01 krb5kdc[8212](info): set up 2 sockets
Aug 08 22:26:40 dist-test-slave-7f01 krb5kdc[8212](info): commencing operation
krb5kdc: starting...
W20260808 22:26:41.592839 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.230s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:26:41 dist-test-slave-7f01 krb5kdc[8212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:26:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:26:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.11.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
I20260808 22:26:41.716711 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:44629
--webserver_interface=127.5.11.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:46143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:44629
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:26:41.829644 8233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:26:41.829885 8233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:26:41.829926 8233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:26:41.833925 8233 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:26:41.833977 8233 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 22:26:41.833994 8233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:26:41.834012 8233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:26:41.834054 8233 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:26:41.834089 8233 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:26:41.839291 8233 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:46143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:44629
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:44629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:26:41.840314 8233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:26:41.841341 8233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:26:41.850054 8249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:41.850265 8256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:41.850332 8233 server_base.cc:1034] running on GCE node
W20260808 22:26:41.850384 8245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:41.850930 8233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:26:41.852041 8233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:26:41.853255 8233 hybrid_clock.cc:648] HybridClock initialized: now 1786228001853233 us; error 31 us; skew 500 ppm
Aug 08 22:26:41 dist-test-slave-7f01 krb5kdc[8212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228001, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:26:41.856025 8233 init.cc:377] Logged in from keytab as kudu/127.5.11.254@KRBTEST.COM (short username kudu)
I20260808 22:26:41.856859 8233 webserver.cc:460] Webserver started at http://127.5.11.254:37787/ using document root <none> and password file <none>
I20260808 22:26:41.857414 8233 fs_manager.cc:359] Metadata directory not provided
I20260808 22:26:41.857522 8233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:26:41.857755 8233 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:26:41.859807 8233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "08a9e7267a0246319859b8084419fecb"
format_stamp: "Formatted at 2026-08-08 22:26:41 on dist-test-slave-7f01"
server_key: "a5e0d0b6b3735c71d39d43a60288983a"
server_key_iv: "5870ba3cfb59d40cfc7827c47252ae12"
server_key_version: "encryptionkey@0"
I20260808 22:26:41.860394 8233 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "08a9e7267a0246319859b8084419fecb"
format_stamp: "Formatted at 2026-08-08 22:26:41 on dist-test-slave-7f01"
server_key: "a5e0d0b6b3735c71d39d43a60288983a"
server_key_iv: "5870ba3cfb59d40cfc7827c47252ae12"
server_key_version: "encryptionkey@0"
I20260808 22:26:41.864059 8233 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:26:41.866479 8271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:41.867633 8233 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:26:41.867820 8233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "08a9e7267a0246319859b8084419fecb"
format_stamp: "Formatted at 2026-08-08 22:26:41 on dist-test-slave-7f01"
server_key: "a5e0d0b6b3735c71d39d43a60288983a"
server_key_iv: "5870ba3cfb59d40cfc7827c47252ae12"
server_key_version: "encryptionkey@0"
I20260808 22:26:41.867945 8233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:26:41.900334 8233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:26:41.903491 8233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:26:41.903818 8233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:26:41.914462 8233 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:44629
I20260808 22:26:41.914471 8394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:44629 every 8 connection(s)
I20260808 22:26:41.915644 8233 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:26:41.918725 8396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:41.923053 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 8233
I20260808 22:26:41.923144 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:26:41.923420 5167 external_mini_cluster.cc:1428] Setting key 8fcafa9c9959765bf9b7698c28a2b210
I20260808 22:26:41.924497 8396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08a9e7267a0246319859b8084419fecb: Bootstrap starting.
I20260808 22:26:41.926844 8396 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 08a9e7267a0246319859b8084419fecb: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:41.927600 8396 log.cc:824] T 00000000000000000000000000000000 P 08a9e7267a0246319859b8084419fecb: Log is configured to *not* fsync() on all Append() calls
I20260808 22:26:41.929484 8396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 08a9e7267a0246319859b8084419fecb: No bootstrap required, opened a new log
Aug 08 22:26:41 dist-test-slave-7f01 krb5kdc[8212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228001, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:26:41.933765 8396 raft_consensus.cc:348] T 00000000000000000000000000000000 P 08a9e7267a0246319859b8084419fecb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08a9e7267a0246319859b8084419fecb" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 44629 } }
I20260808 22:26:41.934023 8396 raft_consensus.cc:374] T 00000000000000000000000000000000 P 08a9e7267a0246319859b8084419fecb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:41.934093 8396 raft_consensus.cc:729] T 00000000000000000000000000000000 P 08a9e7267a0246319859b8084419fecb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08a9e7267a0246319859b8084419fecb, State: Initialized, Role: FOLLOWER
I20260808 22:26:41.934518 8396 consensus_queue.cc:260] T 00000000000000000000000000000000 P 08a9e7267a0246319859b8084419fecb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08a9e7267a0246319859b8084419fecb" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 44629 } }
I20260808 22:26:41.934685 8396 raft_consensus.cc:388] T 00000000000000000000000000000000 P 08a9e7267a0246319859b8084419fecb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:26:41.934770 8396 raft_consensus.cc:482] T 00000000000000000000000000000000 P 08a9e7267a0246319859b8084419fecb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:26:41.934896 8396 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 08a9e7267a0246319859b8084419fecb [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:41.935884 8396 raft_consensus.cc:504] T 00000000000000000000000000000000 P 08a9e7267a0246319859b8084419fecb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08a9e7267a0246319859b8084419fecb" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 44629 } }
I20260808 22:26:41.936236 8396 leader_election.cc:302] T 00000000000000000000000000000000 P 08a9e7267a0246319859b8084419fecb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 08a9e7267a0246319859b8084419fecb; no voters:
I20260808 22:26:41.936579 8396 leader_election.cc:288] T 00000000000000000000000000000000 P 08a9e7267a0246319859b8084419fecb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:26:41.936647 8405 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 08a9e7267a0246319859b8084419fecb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:26:41.936856 8405 raft_consensus.cc:686] T 00000000000000000000000000000000 P 08a9e7267a0246319859b8084419fecb [term 1 LEADER]: Becoming Leader. State: Replica: 08a9e7267a0246319859b8084419fecb, State: Running, Role: LEADER
I20260808 22:26:41.937160 8405 consensus_queue.cc:237] T 00000000000000000000000000000000 P 08a9e7267a0246319859b8084419fecb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08a9e7267a0246319859b8084419fecb" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 44629 } }
I20260808 22:26:41.937835 8396 sys_catalog.cc:564] T 00000000000000000000000000000000 P 08a9e7267a0246319859b8084419fecb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:26:41.938320 8405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08a9e7267a0246319859b8084419fecb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "08a9e7267a0246319859b8084419fecb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08a9e7267a0246319859b8084419fecb" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 44629 } } }
I20260808 22:26:41.938452 8405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08a9e7267a0246319859b8084419fecb [sys.catalog]: This master's current role is: LEADER
I20260808 22:26:41.939147 8407 sys_catalog.cc:455] T 00000000000000000000000000000000 P 08a9e7267a0246319859b8084419fecb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 08a9e7267a0246319859b8084419fecb. Latest consensus state: current_term: 1 leader_uuid: "08a9e7267a0246319859b8084419fecb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "08a9e7267a0246319859b8084419fecb" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 44629 } } }
I20260808 22:26:41.939253 8407 sys_catalog.cc:458] T 00000000000000000000000000000000 P 08a9e7267a0246319859b8084419fecb [sys.catalog]: This master's current role is: LEADER
I20260808 22:26:41.939504 8403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:41.924905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45104 (local address 127.5.11.254:44629)
0808 22:26:41.925326 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:41.925341 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:41.925378 (+ 37us) server_negotiation.cc:407] Connection header received
0808 22:26:41.926018 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:41.926035 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:41.926408 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:41.926740 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:41.927604 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:41.928625 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:41.929506 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:41.930032 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:41.930208 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:41.933132 (+ 2924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:41.933137 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:41.933149 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:41.933174 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:41.935129 (+ 1955us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:41.935636 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:41.935642 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:41.935648 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:41.935974 (+ 326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:41.936306 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:41.936312 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:41.936317 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:41.936801 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:41.937041 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:41.937774 (+ 733us) server_negotiation.cc:299] Negotiation successful
0808 22:26:41.938076 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":125,"threads_started":1}
W20260808 22:26:41.943236 8437 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 08a9e7267a0246319859b8084419fecb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:26:41.943317 8437 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:26:41.943559 8444 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:26:41.944491 8444 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:26:41.949810 8444 catalog_manager.cc:1347] Generated new cluster ID: 2be2351cf11b4f9eb59abc58b6a370ad
I20260808 22:26:41.949877 8444 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:26:41.961503 8444 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:26:41.962477 8444 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:26:41.966811 8444 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 08a9e7267a0246319859b8084419fecb: Generated new TSK 0
I20260808 22:26:41.967470 8444 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.11.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab.
Entry for principal kudu/127.5.11.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab.
I20260808 22:26:42.000561 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.193:0
--local_ip_for_outbound_sockets=127.5.11.193
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:44629
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:46143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:26:42.141292 8449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:26:42.141525 8449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:26:42.141568 8449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:26:42.145432 8449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:26:42.145481 8449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:26:42.145529 8449 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 22:26:42.145542 8449 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 22:26:42.145572 8449 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.193
I20260808 22:26:42.150612 8449 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:46143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.11.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.11.254:44629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.193
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:26:42.151542 8449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:26:42.152539 8449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:26:42.162211 8457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:42.162212 8460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:42.162341 8449 server_base.cc:1034] running on GCE node
W20260808 22:26:42.162209 8462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:42.162974 8449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:26:42.163625 8449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:26:42.164829 8449 hybrid_clock.cc:648] HybridClock initialized: now 1786228002164805 us; error 34 us; skew 500 ppm
Aug 08 22:26:42 dist-test-slave-7f01 krb5kdc[8212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228002, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:26:42.167897 8449 init.cc:377] Logged in from keytab as kudu/127.5.11.193@KRBTEST.COM (short username kudu)
I20260808 22:26:42.168711 8449 webserver.cc:460] Webserver started at http://127.5.11.193:46651/ using document root <none> and password file <none>
I20260808 22:26:42.169240 8449 fs_manager.cc:359] Metadata directory not provided
I20260808 22:26:42.169297 8449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:26:42.169464 8449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:26:42.171583 8449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-0/data/instance:
uuid: "362b664198bb4b25aacfd84c6f0d5119"
format_stamp: "Formatted at 2026-08-08 22:26:42 on dist-test-slave-7f01"
server_key: "1c1c481f240ed976c3908ca6b84a5cc0"
server_key_iv: "7857bce0f399518e9699367bd08375c1"
server_key_version: "encryptionkey@0"
I20260808 22:26:42.172225 8449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance:
uuid: "362b664198bb4b25aacfd84c6f0d5119"
format_stamp: "Formatted at 2026-08-08 22:26:42 on dist-test-slave-7f01"
server_key: "1c1c481f240ed976c3908ca6b84a5cc0"
server_key_iv: "7857bce0f399518e9699367bd08375c1"
server_key_version: "encryptionkey@0"
I20260808 22:26:42.175968 8449 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:26:42.178453 8472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:42.179689 8449 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:26:42.179911 8449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-0/wal
uuid: "362b664198bb4b25aacfd84c6f0d5119"
format_stamp: "Formatted at 2026-08-08 22:26:42 on dist-test-slave-7f01"
server_key: "1c1c481f240ed976c3908ca6b84a5cc0"
server_key_iv: "7857bce0f399518e9699367bd08375c1"
server_key_version: "encryptionkey@0"
I20260808 22:26:42.180006 8449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:26:42.193480 8449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:26:42.196732 8449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:26:42.196954 8449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:26:42.198518 8449 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:26:42.198566 8449 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:42.198606 8449 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:26:42.198660 8449 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 22:26:42 dist-test-slave-7f01 krb5kdc[8212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228002, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:26:42.209703 8449 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.193:44745
I20260808 22:26:42.209777 8680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.193:44745 every 8 connection(s)
I20260808 22:26:42.210688 8449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
I20260808 22:26:42.212718 8403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:42.199262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:56253 (local address 127.5.11.254:44629)
0808 22:26:42.199436 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:42.199443 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:42.200772 (+ 1329us) server_negotiation.cc:407] Connection header received
0808 22:26:42.202142 (+ 1370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:42.202146 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:42.202206 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:42.202284 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:42.203978 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.204557 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.205783 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.206039 (+ 256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.206162 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:42.208794 (+ 2632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:42.208798 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:42.208801 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:42.208836 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:42.210415 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:42.211006 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:42.211009 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:42.211010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:42.211132 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:42.211692 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:42.211694 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:42.211695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:42.211945 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:42.212025 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:42.212483 (+ 458us) server_negotiation.cc:299] Negotiation successful
0808 22:26:42.212597 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 22:26:42.213471 8488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:42.200111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:44629 (local address 127.5.11.193:56253)
0808 22:26:42.200591 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:42.200656 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:42.201889 (+ 1233us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:42.202520 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:42.202533 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:42.203038 (+ 505us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:42.203794 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:42.203819 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.204681 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.204684 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:42.205644 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:42.205655 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.206218 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.206222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:42.206404 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:42.206975 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:26:42.206991 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:42.208621 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:42.210534 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:42.210539 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:42.210552 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:42.210899 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:42.211250 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:42.211253 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:42.211255 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:42.211436 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:42.212039 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:42.212044 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:42.212256 (+ 212us) client_negotiation.cc:769] Sending connection context
0808 22:26:42.212416 (+ 160us) client_negotiation.cc:240] Negotiation successful
0808 22:26:42.212648 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":105,"threads_started":1}
I20260808 22:26:42.214864 8681 heartbeater.cc:347] Connected to a master server at 127.5.11.254:44629
I20260808 22:26:42.215109 8681 heartbeater.cc:464] Registering TS with master...
I20260808 22:26:42.215651 8681 heartbeater.cc:511] Master 127.5.11.254:44629 requested a full tablet report, sending...
I20260808 22:26:42.217161 8305 ts_manager.cc:194] Registered new tserver with Master: 362b664198bb4b25aacfd84c6f0d5119 (127.5.11.193:44745)
I20260808 22:26:42.217911 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 8449
I20260808 22:26:42.218042 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance
I20260808 22:26:42.218418 5167 external_mini_cluster.cc:1428] Setting key 363662350e24f35ce9baa68c926076ea
I20260808 22:26:42.218622 8305 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.193@KRBTEST.COM'} at 127.5.11.193:56253
I20260808 22:26:42.225426 8403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:42.217373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:33655 (local address 127.5.11.254:44629)
0808 22:26:42.217503 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:42.217508 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:42.217787 (+ 279us) server_negotiation.cc:407] Connection header received
0808 22:26:42.217888 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:42.217897 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:42.217951 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:42.218040 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:42.218654 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.219345 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.220838 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.221051 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.221135 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:42.222422 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:42.222426 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:42.222429 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:42.222460 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:42.223817 (+ 1357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:42.224240 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:42.224244 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:42.224245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:42.224328 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:42.224618 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:42.224622 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:42.224624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:42.224795 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:42.224908 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:42.225164 (+ 256us) server_negotiation.cc:299] Negotiation successful
0808 22:26:42.225276 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 22:26:42.225435 8687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:42.217245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:44629 (local address 127.5.11.193:33655)
0808 22:26:42.217687 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:42.217702 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:42.217799 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:42.218173 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:42.218178 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:42.218389 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:42.218525 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:42.218532 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.219856 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.219859 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:42.220718 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:42.220734 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.221171 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.221178 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:42.221303 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:42.221789 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:26:42.221808 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:42.222280 (+ 472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:42.223922 (+ 1642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:42.223925 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:42.223927 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:42.224130 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:42.224422 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:42.224425 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:42.224427 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:42.224523 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:42.224896 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:42.224899 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:42.224997 (+ 98us) client_negotiation.cc:769] Sending connection context
0808 22:26:42.225109 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 22:26:42.225245 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":89,"threads_started":1}
WARNING: no policy specified for kudu/127.5.11.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab.
Entry for principal kudu/127.5.11.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab.
I20260808 22:26:42.246507 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.194:0
--local_ip_for_outbound_sockets=127.5.11.194
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:44629
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:46143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:26:42.357416 8694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:26:42.357702 8694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:26:42.357777 8694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:26:42.361958 8694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:26:42.362031 8694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:26:42.362097 8694 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 22:26:42.362136 8694 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 22:26:42.362202 8694 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.194
I20260808 22:26:42.367384 8694 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:46143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.11.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.11.254:44629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.194
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:26:42.368432 8694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:26:42.369429 8694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:26:42.379217 8709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:42.379238 8705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:42.379241 8706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:42.379315 8694 server_base.cc:1034] running on GCE node
I20260808 22:26:42.379942 8694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:26:42.380587 8694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:26:42.381776 8694 hybrid_clock.cc:648] HybridClock initialized: now 1786228002381753 us; error 32 us; skew 500 ppm
Aug 08 22:26:42 dist-test-slave-7f01 krb5kdc[8212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228002, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:26:42.384615 8694 init.cc:377] Logged in from keytab as kudu/127.5.11.194@KRBTEST.COM (short username kudu)
I20260808 22:26:42.385416 8694 webserver.cc:460] Webserver started at http://127.5.11.194:41613/ using document root <none> and password file <none>
I20260808 22:26:42.385959 8694 fs_manager.cc:359] Metadata directory not provided
I20260808 22:26:42.386059 8694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:26:42.386284 8694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:26:42.388296 8694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-1/data/instance:
uuid: "38ebf0799f9044a688c56196e95a4f8f"
format_stamp: "Formatted at 2026-08-08 22:26:42 on dist-test-slave-7f01"
server_key: "4e984f221ddda4511486fb9d39d51cab"
server_key_iv: "2e25d8bbbd885e54acd64cff60b92355"
server_key_version: "encryptionkey@0"
I20260808 22:26:42.388851 8694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance:
uuid: "38ebf0799f9044a688c56196e95a4f8f"
format_stamp: "Formatted at 2026-08-08 22:26:42 on dist-test-slave-7f01"
server_key: "4e984f221ddda4511486fb9d39d51cab"
server_key_iv: "2e25d8bbbd885e54acd64cff60b92355"
server_key_version: "encryptionkey@0"
I20260808 22:26:42.392414 8694 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 22:26:42.394723 8722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:42.395956 8694 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:26:42.396092 8694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-1/wal
uuid: "38ebf0799f9044a688c56196e95a4f8f"
format_stamp: "Formatted at 2026-08-08 22:26:42 on dist-test-slave-7f01"
server_key: "4e984f221ddda4511486fb9d39d51cab"
server_key_iv: "2e25d8bbbd885e54acd64cff60b92355"
server_key_version: "encryptionkey@0"
I20260808 22:26:42.396198 8694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:26:42.406780 8694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:26:42.409788 8694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:26:42.409982 8694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:26:42.411299 8694 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:26:42.411373 8694 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:42.411443 8694 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:26:42.411494 8694 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 22:26:42 dist-test-slave-7f01 krb5kdc[8212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228002, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:26:42.422523 8694 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.194:36057
I20260808 22:26:42.422653 8928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.194:36057 every 8 connection(s)
I20260808 22:26:42.423514 8694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
I20260808 22:26:42.425429 8403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:42.412128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:58175 (local address 127.5.11.254:44629)
0808 22:26:42.412250 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:42.412254 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:42.413247 (+ 993us) server_negotiation.cc:407] Connection header received
0808 22:26:42.414274 (+ 1027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:42.414280 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:42.414362 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:42.414428 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:42.415973 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.416899 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.418661 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.418991 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.419108 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:42.421933 (+ 2825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:42.421936 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:42.421938 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:42.421962 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:42.423389 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:42.424012 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:42.424015 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:42.424017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:42.424110 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:42.424497 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:42.424501 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:42.424503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:42.424686 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:42.424762 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:42.425216 (+ 454us) server_negotiation.cc:299] Negotiation successful
0808 22:26:42.425311 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 22:26:42.426155 8733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:42.412641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:44629 (local address 127.5.11.194:58175)
0808 22:26:42.413111 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:42.413148 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:42.414004 (+ 856us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:42.414784 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:42.414791 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:42.415133 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:42.415795 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:42.415815 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.417014 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.417016 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:42.418519 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:42.418534 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.419171 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.419175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:42.419437 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:42.420017 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:26:42.420035 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:42.421755 (+ 1720us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:42.423512 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:42.423518 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:42.423532 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:42.423887 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:42.424197 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:42.424200 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:42.424202 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:42.424376 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:42.424777 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:42.424782 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:42.425003 (+ 221us) client_negotiation.cc:769] Sending connection context
0808 22:26:42.425142 (+ 139us) client_negotiation.cc:240] Negotiation successful
0808 22:26:42.425359 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":153,"threads_started":1}
I20260808 22:26:42.427176 8929 heartbeater.cc:347] Connected to a master server at 127.5.11.254:44629
I20260808 22:26:42.427328 8929 heartbeater.cc:464] Registering TS with master...
I20260808 22:26:42.427891 8929 heartbeater.cc:511] Master 127.5.11.254:44629 requested a full tablet report, sending...
I20260808 22:26:42.429037 8305 ts_manager.cc:194] Registered new tserver with Master: 38ebf0799f9044a688c56196e95a4f8f (127.5.11.194:36057)
I20260808 22:26:42.429800 8305 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.194@KRBTEST.COM'} at 127.5.11.194:58175
I20260808 22:26:42.433295 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 8694
I20260808 22:26:42.433422 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance
I20260808 22:26:42.433728 5167 external_mini_cluster.cc:1428] Setting key 64b2650837f78e7b3eacd1b713ff3681
I20260808 22:26:42.436745 8403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:42.429519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:37473 (local address 127.5.11.254:44629)
0808 22:26:42.429632 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:42.429636 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:42.429884 (+ 248us) server_negotiation.cc:407] Connection header received
0808 22:26:42.429972 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:42.429975 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:42.430021 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:42.430100 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:42.430615 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.431340 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.432169 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.432376 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.432462 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:42.433600 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:42.433603 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:42.433605 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:42.433624 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:42.435056 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:42.435470 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:42.435474 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:42.435476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:42.435563 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:42.435918 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:42.435921 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:42.435922 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:42.436129 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:42.436243 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:42.436509 (+ 266us) server_negotiation.cc:299] Negotiation successful
0808 22:26:42.436614 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 22:26:42.436757 8937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:42.429449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:44629 (local address 127.5.11.194:37473)
0808 22:26:42.429766 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:42.429778 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:42.429892 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:42.430165 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:42.430168 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:42.430382 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:42.430498 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:42.430504 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.431456 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.431458 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:42.432065 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:42.432074 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.432470 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.432472 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:42.432548 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:42.433018 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:26:42.433064 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:42.433460 (+ 396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:42.435160 (+ 1700us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:42.435165 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:42.435167 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:42.435368 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:42.435656 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:42.435660 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:42.435662 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:42.435803 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:42.436235 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:42.436238 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:42.436334 (+ 96us) client_negotiation.cc:769] Sending connection context
0808 22:26:42.436451 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 22:26:42.436580 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":94,"threads_started":1}
WARNING: no policy specified for kudu/127.5.11.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab.
Entry for principal kudu/127.5.11.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab.
I20260808 22:26:42.462359 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.195:0
--local_ip_for_outbound_sockets=127.5.11.195
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:44629
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:46143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:26:42.572551 8943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:26:42.572829 8943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:26:42.572907 8943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:26:42.577059 8943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:26:42.577142 8943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:26:42.577206 8943 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260808 22:26:42.577248 8943 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 22:26:42.577322 8943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.195
I20260808 22:26:42.582438 8943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:46143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.11.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.11.254:44629
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.195
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:26:42.583460 8943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:26:42.584561 8943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:26:42.593873 8958 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:42.593886 8957 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:42.593890 8963 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:42.594079 8943 server_base.cc:1034] running on GCE node
I20260808 22:26:42.594519 8943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:26:42.595116 8943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:26:42.596288 8943 hybrid_clock.cc:648] HybridClock initialized: now 1786228002596265 us; error 35 us; skew 500 ppm
Aug 08 22:26:42 dist-test-slave-7f01 krb5kdc[8212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228002, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:26:42.599084 8943 init.cc:377] Logged in from keytab as kudu/127.5.11.195@KRBTEST.COM (short username kudu)
I20260808 22:26:42.599977 8943 webserver.cc:460] Webserver started at http://127.5.11.195:40933/ using document root <none> and password file <none>
I20260808 22:26:42.600488 8943 fs_manager.cc:359] Metadata directory not provided
I20260808 22:26:42.600546 8943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:26:42.600703 8943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:26:42.602600 8943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-2/data/instance:
uuid: "b016084d0efc4659b75da627a2c14a6e"
format_stamp: "Formatted at 2026-08-08 22:26:42 on dist-test-slave-7f01"
server_key: "b60c0232a510a4a1a2a8842c18a2f109"
server_key_iv: "815fd041929193976d0d09a6f71de32b"
server_key_version: "encryptionkey@0"
I20260808 22:26:42.603091 8943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance:
uuid: "b016084d0efc4659b75da627a2c14a6e"
format_stamp: "Formatted at 2026-08-08 22:26:42 on dist-test-slave-7f01"
server_key: "b60c0232a510a4a1a2a8842c18a2f109"
server_key_iv: "815fd041929193976d0d09a6f71de32b"
server_key_version: "encryptionkey@0"
I20260808 22:26:42.606796 8943 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 22:26:42.609206 8973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:42.610400 8943 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:26:42.610527 8943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-2/wal
uuid: "b016084d0efc4659b75da627a2c14a6e"
format_stamp: "Formatted at 2026-08-08 22:26:42 on dist-test-slave-7f01"
server_key: "b60c0232a510a4a1a2a8842c18a2f109"
server_key_iv: "815fd041929193976d0d09a6f71de32b"
server_key_version: "encryptionkey@0"
I20260808 22:26:42.610630 8943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:26:42.627532 8943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:26:42.630914 8943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:26:42.631117 8943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:26:42.632611 8943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:26:42.632659 8943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:42.632725 8943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:26:42.632761 8943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 22:26:42 dist-test-slave-7f01 krb5kdc[8212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228002, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:26:42.643922 8943 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.195:41455
I20260808 22:26:42.644104 9183 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.195:41455 every 8 connection(s)
I20260808 22:26:42.644807 8943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
I20260808 22:26:42.646704 8403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:42.633450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:43039 (local address 127.5.11.254:44629)
0808 22:26:42.633641 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:42.633647 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:42.634396 (+ 749us) server_negotiation.cc:407] Connection header received
0808 22:26:42.635557 (+ 1161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:42.635560 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:42.635636 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:42.635703 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:42.637145 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.637688 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.639490 (+ 1802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.639887 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.640019 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:42.642901 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:42.642906 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:42.642910 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:42.642940 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:42.644581 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:42.645162 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:42.645167 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:42.645169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:42.645281 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:42.645648 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:42.645653 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:42.645655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:42.645896 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:42.645991 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:42.646481 (+ 490us) server_negotiation.cc:299] Negotiation successful
0808 22:26:42.646583 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 22:26:42.647480 8988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:42.633789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:44629 (local address 127.5.11.195:43039)
0808 22:26:42.634224 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:42.634274 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:42.635288 (+ 1014us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:42.636093 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:42.636099 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:42.636406 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:42.636975 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:42.636997 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.637841 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.637845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:42.639378 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:42.639391 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.640065 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.640069 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:42.640241 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:42.640823 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:26:42.640841 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:42.642729 (+ 1888us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:42.644695 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:42.644701 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:42.644715 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:42.645035 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:42.645397 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:42.645400 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:42.645402 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:42.645561 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:42.646029 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:42.646034 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:42.646247 (+ 213us) client_negotiation.cc:769] Sending connection context
0808 22:26:42.646413 (+ 166us) client_negotiation.cc:240] Negotiation successful
0808 22:26:42.646644 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":133,"threads_started":1}
I20260808 22:26:42.648473 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 8943
I20260808 22:26:42.648592 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance
I20260808 22:26:42.648857 9184 heartbeater.cc:347] Connected to a master server at 127.5.11.254:44629
I20260808 22:26:42.648917 5167 external_mini_cluster.cc:1428] Setting key 9c2628188f3a8e8b8882ae063288db23
I20260808 22:26:42.649034 9184 heartbeater.cc:464] Registering TS with master...
I20260808 22:26:42.649636 9184 heartbeater.cc:511] Master 127.5.11.254:44629 requested a full tablet report, sending...
I20260808 22:26:42.650635 8305 ts_manager.cc:194] Registered new tserver with Master: b016084d0efc4659b75da627a2c14a6e (127.5.11.195:41455)
I20260808 22:26:42.651417 8305 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.195@KRBTEST.COM'} at 127.5.11.195:43039
I20260808 22:26:42.657907 8403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:42.651128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:37025 (local address 127.5.11.254:44629)
0808 22:26:42.651249 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:42.651253 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:42.651412 (+ 159us) server_negotiation.cc:407] Connection header received
0808 22:26:42.651482 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:42.651486 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:42.651528 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:42.651609 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:42.652093 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.652591 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.653389 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.653627 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.653728 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:42.654753 (+ 1025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:42.654757 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:42.654760 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:42.654783 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:42.656292 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:42.656731 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:42.656736 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:42.656737 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:42.656818 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:42.657117 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:42.657120 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:42.657122 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:42.657292 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:42.657394 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:42.657654 (+ 260us) server_negotiation.cc:299] Negotiation successful
0808 22:26:42.657762 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 22:26:42.657920 9195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:42.651080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:44629 (local address 127.5.11.195:37025)
0808 22:26:42.651322 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:42.651344 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:42.651419 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:42.651624 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:42.651626 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:42.651880 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:42.651979 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:42.651985 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.652711 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.652715 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:42.653270 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:42.653278 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.653741 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.653744 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:42.653840 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:42.654320 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:26:42.654331 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:42.654620 (+ 289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:42.656421 (+ 1801us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:42.656425 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:42.656426 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:42.656631 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:42.656920 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:42.656923 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:42.656925 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:42.657016 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:42.657385 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:42.657389 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:42.657486 (+ 97us) client_negotiation.cc:769] Sending connection context
0808 22:26:42.657595 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 22:26:42.657729 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":93,"threads_started":1}
I20260808 22:26:42.662387 5167 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 22:26:42 dist-test-slave-7f01 krb5kdc[8212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 22:26:42 dist-test-slave-7f01 krb5kdc[8212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228002, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:26:42.689041 8403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:42.680842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45120 (local address 127.5.11.254:44629)
0808 22:26:42.680976 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:42.680979 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:42.680992 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:26:42.681027 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:42.681029 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:42.681064 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:42.681153 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:42.681841 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.682344 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.683265 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.683477 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.683592 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:42.685958 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:42.685966 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:42.685969 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:42.685992 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:42.687393 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:42.687847 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:42.687851 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:42.687853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:42.687932 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:42.688216 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:42.688220 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:42.688222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:42.688391 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:42.688539 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:42.688766 (+ 227us) server_negotiation.cc:299] Negotiation successful
0808 22:26:42.688882 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 22:26:42.691915 8305 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45120:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:26:42.694854 8305 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:26:42.708959 9215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:42.702804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:44745 (local address 127.0.0.1:58640)
0808 22:26:42.703641 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:42.703689 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:42.703828 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:42.704945 (+ 1117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:42.704950 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:42.704972 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:26:42.705060 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:42.705067 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.706101 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.706107 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:42.707365 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:42.707375 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.708397 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.708401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:42.708524 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:42.708578 (+ 54us) client_negotiation.cc:769] Sending connection context
0808 22:26:42.708683 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 22:26:42.708810 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"mutex_wait_us":52,"thread_start_us":100,"threads_started":1}
I20260808 22:26:42.709282 9221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:42.703120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58640 (local address 127.5.11.193:44745)
0808 22:26:42.704402 (+ 1282us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:42.704416 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:42.704434 (+ 18us) server_negotiation.cc:407] Connection header received
0808 22:26:42.704495 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:42.704499 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:42.704630 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:42.704802 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:26:42.705191 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.705981 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.707497 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.708258 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.708374 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:42.708544 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:42.709046 (+ 502us) server_negotiation.cc:299] Negotiation successful
0808 22:26:42.709146 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1193,"thread_start_us":71,"threads_started":1}
I20260808 22:26:42.709270 9216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:42.702912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:41455 (local address 127.0.0.1:41954)
0808 22:26:42.704074 (+ 1162us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:42.704086 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:42.704198 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:42.704726 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:42.704735 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:42.704764 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:26:42.704886 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:42.704894 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.706364 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.706367 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:42.707646 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:42.707658 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.708898 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.708903 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:42.708994 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:42.709019 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 22:26:42.709091 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 22:26:42.709146 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1093,"mutex_wait_us":1,"thread_start_us":84,"threads_started":1}
I20260808 22:26:42.709576 9224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:42.703824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50598 (local address 127.5.11.194:36057)
0808 22:26:42.704673 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:42.704685 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:42.704705 (+ 20us) server_negotiation.cc:407] Connection header received
0808 22:26:42.704773 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:42.704777 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:42.704937 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:42.705114 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:26:42.705330 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.706399 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.707856 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.708461 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.708575 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:42.708732 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:42.708875 (+ 143us) server_negotiation.cc:299] Negotiation successful
0808 22:26:42.708967 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":739,"thread_start_us":157,"threads_started":1}
I20260808 22:26:42.709769 9217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:42.703056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41954 (local address 127.5.11.195:41455)
0808 22:26:42.703535 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:42.703549 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:42.704192 (+ 643us) server_negotiation.cc:407] Connection header received
0808 22:26:42.704318 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:42.704323 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:42.704551 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:42.704710 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:26:42.705016 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.706234 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.707804 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.708584 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.708688 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:42.708840 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:42.709506 (+ 666us) server_negotiation.cc:299] Negotiation successful
0808 22:26:42.709618 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":110,"threads_started":1}
I20260808 22:26:42.710108 9222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:42.703586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:36057 (local address 127.0.0.1:50598)
0808 22:26:42.704332 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:42.704344 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:42.704428 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:42.705095 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:42.705101 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:42.705123 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:26:42.705213 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:42.705219 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.706513 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.706517 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:42.707688 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:42.707698 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.708633 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.708640 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:42.708731 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:42.708756 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 22:26:42.708843 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 22:26:42.708934 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":685,"mutex_wait_us":22,"thread_start_us":93,"threads_started":1}
I20260808 22:26:42.712749 9069 tablet_service.cc:1467] Processing CreateTablet for tablet debe1d8bd94c49cea47f4ffff243dac4 (DEFAULT_TABLE table=test-table [id=5befd864b1c14a29a1e8be0505f29dd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:26:42.712750 8559 tablet_service.cc:1467] Processing CreateTablet for tablet debe1d8bd94c49cea47f4ffff243dac4 (DEFAULT_TABLE table=test-table [id=5befd864b1c14a29a1e8be0505f29dd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:26:42.713930 9069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet debe1d8bd94c49cea47f4ffff243dac4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:42.713930 8559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet debe1d8bd94c49cea47f4ffff243dac4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:42.714303 8803 tablet_service.cc:1467] Processing CreateTablet for tablet debe1d8bd94c49cea47f4ffff243dac4 (DEFAULT_TABLE table=test-table [id=5befd864b1c14a29a1e8be0505f29dd2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:26:42.715304 8803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet debe1d8bd94c49cea47f4ffff243dac4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:42.721755 9227 tablet_bootstrap.cc:492] T debe1d8bd94c49cea47f4ffff243dac4 P 362b664198bb4b25aacfd84c6f0d5119: Bootstrap starting.
I20260808 22:26:42.722424 9228 tablet_bootstrap.cc:492] T debe1d8bd94c49cea47f4ffff243dac4 P 38ebf0799f9044a688c56196e95a4f8f: Bootstrap starting.
I20260808 22:26:42.723268 9229 tablet_bootstrap.cc:492] T debe1d8bd94c49cea47f4ffff243dac4 P b016084d0efc4659b75da627a2c14a6e: Bootstrap starting.
I20260808 22:26:42.724471 9228 tablet_bootstrap.cc:654] T debe1d8bd94c49cea47f4ffff243dac4 P 38ebf0799f9044a688c56196e95a4f8f: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:42.724578 9227 tablet_bootstrap.cc:654] T debe1d8bd94c49cea47f4ffff243dac4 P 362b664198bb4b25aacfd84c6f0d5119: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:42.725229 9229 tablet_bootstrap.cc:654] T debe1d8bd94c49cea47f4ffff243dac4 P b016084d0efc4659b75da627a2c14a6e: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:42.725227 9228 log.cc:824] T debe1d8bd94c49cea47f4ffff243dac4 P 38ebf0799f9044a688c56196e95a4f8f: Log is configured to *not* fsync() on all Append() calls
I20260808 22:26:42.725405 9227 log.cc:824] T debe1d8bd94c49cea47f4ffff243dac4 P 362b664198bb4b25aacfd84c6f0d5119: Log is configured to *not* fsync() on all Append() calls
I20260808 22:26:42.726028 9229 log.cc:824] T debe1d8bd94c49cea47f4ffff243dac4 P b016084d0efc4659b75da627a2c14a6e: Log is configured to *not* fsync() on all Append() calls
I20260808 22:26:42.727532 9228 tablet_bootstrap.cc:492] T debe1d8bd94c49cea47f4ffff243dac4 P 38ebf0799f9044a688c56196e95a4f8f: No bootstrap required, opened a new log
I20260808 22:26:42.727780 9228 ts_tablet_manager.cc:1397] T debe1d8bd94c49cea47f4ffff243dac4 P 38ebf0799f9044a688c56196e95a4f8f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 22:26:42.727901 9227 tablet_bootstrap.cc:492] T debe1d8bd94c49cea47f4ffff243dac4 P 362b664198bb4b25aacfd84c6f0d5119: No bootstrap required, opened a new log
I20260808 22:26:42.728053 9229 tablet_bootstrap.cc:492] T debe1d8bd94c49cea47f4ffff243dac4 P b016084d0efc4659b75da627a2c14a6e: No bootstrap required, opened a new log
I20260808 22:26:42.728129 9227 ts_tablet_manager.cc:1397] T debe1d8bd94c49cea47f4ffff243dac4 P 362b664198bb4b25aacfd84c6f0d5119: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 22:26:42.728242 9229 ts_tablet_manager.cc:1397] T debe1d8bd94c49cea47f4ffff243dac4 P b016084d0efc4659b75da627a2c14a6e: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 22:26:42.730829 9227 raft_consensus.cc:348] T debe1d8bd94c49cea47f4ffff243dac4 P 362b664198bb4b25aacfd84c6f0d5119 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } }
I20260808 22:26:42.731068 9227 raft_consensus.cc:374] T debe1d8bd94c49cea47f4ffff243dac4 P 362b664198bb4b25aacfd84c6f0d5119 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:42.731133 9227 raft_consensus.cc:729] T debe1d8bd94c49cea47f4ffff243dac4 P 362b664198bb4b25aacfd84c6f0d5119 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 362b664198bb4b25aacfd84c6f0d5119, State: Initialized, Role: FOLLOWER
I20260808 22:26:42.731570 9227 consensus_queue.cc:260] T debe1d8bd94c49cea47f4ffff243dac4 P 362b664198bb4b25aacfd84c6f0d5119 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } }
I20260808 22:26:42.731977 9229 raft_consensus.cc:348] T debe1d8bd94c49cea47f4ffff243dac4 P b016084d0efc4659b75da627a2c14a6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } }
I20260808 22:26:42.731977 9228 raft_consensus.cc:348] T debe1d8bd94c49cea47f4ffff243dac4 P 38ebf0799f9044a688c56196e95a4f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } }
I20260808 22:26:42.732187 8681 heartbeater.cc:503] Master 127.5.11.254:44629 was elected leader, sending a full tablet report...
I20260808 22:26:42.732234 9228 raft_consensus.cc:374] T debe1d8bd94c49cea47f4ffff243dac4 P 38ebf0799f9044a688c56196e95a4f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:42.732234 9229 raft_consensus.cc:374] T debe1d8bd94c49cea47f4ffff243dac4 P b016084d0efc4659b75da627a2c14a6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:42.732286 9228 raft_consensus.cc:729] T debe1d8bd94c49cea47f4ffff243dac4 P 38ebf0799f9044a688c56196e95a4f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38ebf0799f9044a688c56196e95a4f8f, State: Initialized, Role: FOLLOWER
I20260808 22:26:42.732286 9229 raft_consensus.cc:729] T debe1d8bd94c49cea47f4ffff243dac4 P b016084d0efc4659b75da627a2c14a6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b016084d0efc4659b75da627a2c14a6e, State: Initialized, Role: FOLLOWER
I20260808 22:26:42.732704 9229 consensus_queue.cc:260] T debe1d8bd94c49cea47f4ffff243dac4 P b016084d0efc4659b75da627a2c14a6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } }
I20260808 22:26:42.732704 9228 consensus_queue.cc:260] T debe1d8bd94c49cea47f4ffff243dac4 P 38ebf0799f9044a688c56196e95a4f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } }
I20260808 22:26:42.733439 9227 ts_tablet_manager.cc:1428] T debe1d8bd94c49cea47f4ffff243dac4 P 362b664198bb4b25aacfd84c6f0d5119: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 22:26:42.733640 9229 ts_tablet_manager.cc:1428] T debe1d8bd94c49cea47f4ffff243dac4 P b016084d0efc4659b75da627a2c14a6e: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 22:26:42.733640 9228 ts_tablet_manager.cc:1428] T debe1d8bd94c49cea47f4ffff243dac4 P 38ebf0799f9044a688c56196e95a4f8f: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 22:26:42.733973 9184 heartbeater.cc:503] Master 127.5.11.254:44629 was elected leader, sending a full tablet report...
I20260808 22:26:42.735013 8929 heartbeater.cc:503] Master 127.5.11.254:44629 was elected leader, sending a full tablet report...
W20260808 22:26:42.895905 9186 tablet.cc:2369] T debe1d8bd94c49cea47f4ffff243dac4 P b016084d0efc4659b75da627a2c14a6e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:26:42.910184 9243 raft_consensus.cc:482] T debe1d8bd94c49cea47f4ffff243dac4 P 38ebf0799f9044a688c56196e95a4f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:26:42.910430 9243 raft_consensus.cc:504] T debe1d8bd94c49cea47f4ffff243dac4 P 38ebf0799f9044a688c56196e95a4f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } }
I20260808 22:26:42.911623 9243 leader_election.cc:288] T debe1d8bd94c49cea47f4ffff243dac4 P 38ebf0799f9044a688c56196e95a4f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 362b664198bb4b25aacfd84c6f0d5119 (127.5.11.193:44745), b016084d0efc4659b75da627a2c14a6e (127.5.11.195:41455)
I20260808 22:26:42.916431 9221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:42.912148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:36419 (local address 127.5.11.193:44745)
0808 22:26:42.912270 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:42.912276 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:42.912294 (+ 18us) server_negotiation.cc:407] Connection header received
0808 22:26:42.912348 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:42.912350 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:42.912391 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:42.912466 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:26:42.912784 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.913603 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.915188 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.915857 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.915976 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:42.916036 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:42.916234 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 22:26:42.916306 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 22:26:42.916432 8733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:42.911936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:44745 (local address 127.5.11.194:36419)
0808 22:26:42.912047 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:42.912065 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:42.912171 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:42.912511 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:42.912514 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:42.912528 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:26:42.912671 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:42.912676 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.913717 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.913720 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:42.915065 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:42.915078 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.915982 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.915985 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:42.916095 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:42.916122 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 22:26:42.916205 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 22:26:42.916276 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 22:26:42.916946 9217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:42.912064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:43209 (local address 127.5.11.195:41455)
0808 22:26:42.912221 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:42.912226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:42.912520 (+ 294us) server_negotiation.cc:407] Connection header received
0808 22:26:42.912630 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:42.912633 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:42.912675 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:42.912766 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:26:42.913026 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.913787 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.915647 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.916249 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.916334 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:42.916379 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:42.916755 (+ 376us) server_negotiation.cc:299] Negotiation successful
0808 22:26:42.916834 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260808 22:26:42.916952 9246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:42.911936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:41455 (local address 127.5.11.194:43209)
0808 22:26:42.912388 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:42.912402 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:42.912523 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:42.912780 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:42.912783 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:42.912811 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:26:42.912908 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:42.912912 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.913917 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.913921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:42.915461 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:42.915470 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.916491 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.916494 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:42.916602 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:42.916637 (+ 35us) client_negotiation.cc:769] Sending connection context
0808 22:26:42.916715 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 22:26:42.916802 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":395,"mutex_wait_us":121,"thread_start_us":74,"threads_started":1}
I20260808 22:26:42.917467 9111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "debe1d8bd94c49cea47f4ffff243dac4" candidate_uuid: "38ebf0799f9044a688c56196e95a4f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b016084d0efc4659b75da627a2c14a6e" is_pre_election: true
I20260808 22:26:42.917470 8592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "debe1d8bd94c49cea47f4ffff243dac4" candidate_uuid: "38ebf0799f9044a688c56196e95a4f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "362b664198bb4b25aacfd84c6f0d5119" is_pre_election: true
I20260808 22:26:42.917784 8592 raft_consensus.cc:2454] T debe1d8bd94c49cea47f4ffff243dac4 P 362b664198bb4b25aacfd84c6f0d5119 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38ebf0799f9044a688c56196e95a4f8f in term 0.
I20260808 22:26:42.917771 9111 raft_consensus.cc:2454] T debe1d8bd94c49cea47f4ffff243dac4 P b016084d0efc4659b75da627a2c14a6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38ebf0799f9044a688c56196e95a4f8f in term 0.
I20260808 22:26:42.918293 8725 leader_election.cc:302] T debe1d8bd94c49cea47f4ffff243dac4 P 38ebf0799f9044a688c56196e95a4f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 362b664198bb4b25aacfd84c6f0d5119, 38ebf0799f9044a688c56196e95a4f8f, b016084d0efc4659b75da627a2c14a6e; no voters:
I20260808 22:26:42.918715 9243 raft_consensus.cc:2781] T debe1d8bd94c49cea47f4ffff243dac4 P 38ebf0799f9044a688c56196e95a4f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:26:42.918782 9243 raft_consensus.cc:482] T debe1d8bd94c49cea47f4ffff243dac4 P 38ebf0799f9044a688c56196e95a4f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:26:42.918838 9243 raft_consensus.cc:3037] T debe1d8bd94c49cea47f4ffff243dac4 P 38ebf0799f9044a688c56196e95a4f8f [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:42.919953 9243 raft_consensus.cc:504] T debe1d8bd94c49cea47f4ffff243dac4 P 38ebf0799f9044a688c56196e95a4f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } }
I20260808 22:26:42.920606 9243 leader_election.cc:288] T debe1d8bd94c49cea47f4ffff243dac4 P 38ebf0799f9044a688c56196e95a4f8f [CANDIDATE]: Term 1 election: Requested vote from peers 362b664198bb4b25aacfd84c6f0d5119 (127.5.11.193:44745), b016084d0efc4659b75da627a2c14a6e (127.5.11.195:41455)
I20260808 22:26:42.920657 8592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "debe1d8bd94c49cea47f4ffff243dac4" candidate_uuid: "38ebf0799f9044a688c56196e95a4f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "362b664198bb4b25aacfd84c6f0d5119"
I20260808 22:26:42.920779 8592 raft_consensus.cc:3037] T debe1d8bd94c49cea47f4ffff243dac4 P 362b664198bb4b25aacfd84c6f0d5119 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:42.921802 8592 raft_consensus.cc:2454] T debe1d8bd94c49cea47f4ffff243dac4 P 362b664198bb4b25aacfd84c6f0d5119 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38ebf0799f9044a688c56196e95a4f8f in term 1.
I20260808 22:26:42.922107 9111 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "debe1d8bd94c49cea47f4ffff243dac4" candidate_uuid: "38ebf0799f9044a688c56196e95a4f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b016084d0efc4659b75da627a2c14a6e"
I20260808 22:26:42.922159 8724 leader_election.cc:302] T debe1d8bd94c49cea47f4ffff243dac4 P 38ebf0799f9044a688c56196e95a4f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 362b664198bb4b25aacfd84c6f0d5119, 38ebf0799f9044a688c56196e95a4f8f; no voters:
I20260808 22:26:42.922264 9111 raft_consensus.cc:3037] T debe1d8bd94c49cea47f4ffff243dac4 P b016084d0efc4659b75da627a2c14a6e [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:42.922318 9243 raft_consensus.cc:2781] T debe1d8bd94c49cea47f4ffff243dac4 P 38ebf0799f9044a688c56196e95a4f8f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:26:42.922521 9243 raft_consensus.cc:686] T debe1d8bd94c49cea47f4ffff243dac4 P 38ebf0799f9044a688c56196e95a4f8f [term 1 LEADER]: Becoming Leader. State: Replica: 38ebf0799f9044a688c56196e95a4f8f, State: Running, Role: LEADER
I20260808 22:26:42.922890 9243 consensus_queue.cc:237] T debe1d8bd94c49cea47f4ffff243dac4 P 38ebf0799f9044a688c56196e95a4f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } }
I20260808 22:26:42.924113 9111 raft_consensus.cc:2454] T debe1d8bd94c49cea47f4ffff243dac4 P b016084d0efc4659b75da627a2c14a6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38ebf0799f9044a688c56196e95a4f8f in term 1.
W20260808 22:26:42.924803 8930 tablet.cc:2369] T debe1d8bd94c49cea47f4ffff243dac4 P 38ebf0799f9044a688c56196e95a4f8f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:26:42.926025 8304 catalog_manager.cc:5495] T debe1d8bd94c49cea47f4ffff243dac4 P 38ebf0799f9044a688c56196e95a4f8f reported cstate change: term changed from 0 to 1, leader changed from <none> to 38ebf0799f9044a688c56196e95a4f8f (127.5.11.194). New cstate: current_term: 1 leader_uuid: "38ebf0799f9044a688c56196e95a4f8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } health_report { overall_health: HEALTHY } } }
Aug 08 22:26:42 dist-test-slave-7f01 krb5kdc[8212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228001, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:26:42.961464 9255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:42.953088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:44629 (local address 127.0.0.1:45132)
0808 22:26:42.953376 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:42.953387 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:42.953510 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:42.953764 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:42.953767 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:42.954150 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:42.954231 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:42.954237 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.954937 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.954943 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:42.955468 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:42.955475 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.955986 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.955989 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:42.956072 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:42.956690 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:26:42.956702 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:42.958124 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:42.959868 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:42.959878 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:42.959885 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:42.960114 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:42.960384 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:42.960388 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:42.960390 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:42.960474 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:42.960886 (+ 412us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:42.960893 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:42.961012 (+ 119us) client_negotiation.cc:769] Sending connection context
0808 22:26:42.961130 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 22:26:42.961279 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":103,"threads_started":1}
I20260808 22:26:42.961463 8403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:42.953144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45132 (local address 127.5.11.254:44629)
0808 22:26:42.953285 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:42.953291 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:42.953461 (+ 170us) server_negotiation.cc:407] Connection header received
0808 22:26:42.953610 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:42.953614 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:42.953658 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:42.953740 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:42.954352 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.954833 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.955593 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:42.955878 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:42.955984 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:42.958269 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:42.958273 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:42.958276 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:42.958298 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:42.959704 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:42.960211 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:42.960216 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:42.960218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:42.960295 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:42.960573 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:42.960576 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:42.960578 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:42.960763 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:42.960886 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:42.961178 (+ 292us) server_negotiation.cc:299] Negotiation successful
0808 22:26:42.961336 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":44}
W20260808 22:26:42.962106 8682 tablet.cc:2369] T debe1d8bd94c49cea47f4ffff243dac4 P 362b664198bb4b25aacfd84c6f0d5119: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:26:42.965315 8304 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:45132:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 22:26:42.965982 8304 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:26:42.970763 8559 tablet_service.cc:1467] Processing CreateTablet for tablet c54e475da85948da9fca4001755162b4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dd85a4a2b3774e7aaa4e88c3d2fd5893]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 22:26:42.970944 9069 tablet_service.cc:1467] Processing CreateTablet for tablet c54e475da85948da9fca4001755162b4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dd85a4a2b3774e7aaa4e88c3d2fd5893]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 22:26:42.970850 8803 tablet_service.cc:1467] Processing CreateTablet for tablet c54e475da85948da9fca4001755162b4 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=dd85a4a2b3774e7aaa4e88c3d2fd5893]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 22:26:42.971560 9069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c54e475da85948da9fca4001755162b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:42.971562 8559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c54e475da85948da9fca4001755162b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:42.971570 8803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c54e475da85948da9fca4001755162b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:42.974046 9227 tablet_bootstrap.cc:492] T c54e475da85948da9fca4001755162b4 P 362b664198bb4b25aacfd84c6f0d5119: Bootstrap starting.
I20260808 22:26:42.974480 9228 tablet_bootstrap.cc:492] T c54e475da85948da9fca4001755162b4 P 38ebf0799f9044a688c56196e95a4f8f: Bootstrap starting.
I20260808 22:26:42.974643 9229 tablet_bootstrap.cc:492] T c54e475da85948da9fca4001755162b4 P b016084d0efc4659b75da627a2c14a6e: Bootstrap starting.
I20260808 22:26:42.975028 9227 tablet_bootstrap.cc:654] T c54e475da85948da9fca4001755162b4 P 362b664198bb4b25aacfd84c6f0d5119: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:42.975462 9228 tablet_bootstrap.cc:654] T c54e475da85948da9fca4001755162b4 P 38ebf0799f9044a688c56196e95a4f8f: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:42.975556 9229 tablet_bootstrap.cc:654] T c54e475da85948da9fca4001755162b4 P b016084d0efc4659b75da627a2c14a6e: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:42.976438 9227 tablet_bootstrap.cc:492] T c54e475da85948da9fca4001755162b4 P 362b664198bb4b25aacfd84c6f0d5119: No bootstrap required, opened a new log
I20260808 22:26:42.976549 9227 ts_tablet_manager.cc:1397] T c54e475da85948da9fca4001755162b4 P 362b664198bb4b25aacfd84c6f0d5119: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:26:42.976933 9229 tablet_bootstrap.cc:492] T c54e475da85948da9fca4001755162b4 P b016084d0efc4659b75da627a2c14a6e: No bootstrap required, opened a new log
I20260808 22:26:42.977042 9229 ts_tablet_manager.cc:1397] T c54e475da85948da9fca4001755162b4 P b016084d0efc4659b75da627a2c14a6e: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260808 22:26:42.977056 9227 raft_consensus.cc:348] T c54e475da85948da9fca4001755162b4 P 362b664198bb4b25aacfd84c6f0d5119 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } }
I20260808 22:26:42.977186 9227 raft_consensus.cc:374] T c54e475da85948da9fca4001755162b4 P 362b664198bb4b25aacfd84c6f0d5119 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:42.977228 9227 raft_consensus.cc:729] T c54e475da85948da9fca4001755162b4 P 362b664198bb4b25aacfd84c6f0d5119 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 362b664198bb4b25aacfd84c6f0d5119, State: Initialized, Role: FOLLOWER
I20260808 22:26:42.977284 9228 tablet_bootstrap.cc:492] T c54e475da85948da9fca4001755162b4 P 38ebf0799f9044a688c56196e95a4f8f: No bootstrap required, opened a new log
I20260808 22:26:42.977375 9228 ts_tablet_manager.cc:1397] T c54e475da85948da9fca4001755162b4 P 38ebf0799f9044a688c56196e95a4f8f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:26:42.977380 9227 consensus_queue.cc:260] T c54e475da85948da9fca4001755162b4 P 362b664198bb4b25aacfd84c6f0d5119 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } }
I20260808 22:26:42.977536 9229 raft_consensus.cc:348] T c54e475da85948da9fca4001755162b4 P b016084d0efc4659b75da627a2c14a6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } }
I20260808 22:26:42.977663 9229 raft_consensus.cc:374] T c54e475da85948da9fca4001755162b4 P b016084d0efc4659b75da627a2c14a6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:42.977703 9229 raft_consensus.cc:729] T c54e475da85948da9fca4001755162b4 P b016084d0efc4659b75da627a2c14a6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b016084d0efc4659b75da627a2c14a6e, State: Initialized, Role: FOLLOWER
I20260808 22:26:42.977684 9242 tablet_replica.cc:439] T c54e475da85948da9fca4001755162b4 P 362b664198bb4b25aacfd84c6f0d5119: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } } }
I20260808 22:26:42.977811 9242 tablet_replica.cc:442] T c54e475da85948da9fca4001755162b4 P 362b664198bb4b25aacfd84c6f0d5119: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:26:42.977823 9229 consensus_queue.cc:260] T c54e475da85948da9fca4001755162b4 P b016084d0efc4659b75da627a2c14a6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } }
I20260808 22:26:42.977851 9228 raft_consensus.cc:348] T c54e475da85948da9fca4001755162b4 P 38ebf0799f9044a688c56196e95a4f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } }
I20260808 22:26:42.977950 9228 raft_consensus.cc:374] T c54e475da85948da9fca4001755162b4 P 38ebf0799f9044a688c56196e95a4f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:42.977981 9228 raft_consensus.cc:729] T c54e475da85948da9fca4001755162b4 P 38ebf0799f9044a688c56196e95a4f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38ebf0799f9044a688c56196e95a4f8f, State: Initialized, Role: FOLLOWER
I20260808 22:26:42.978093 9228 consensus_queue.cc:260] T c54e475da85948da9fca4001755162b4 P 38ebf0799f9044a688c56196e95a4f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } }
I20260808 22:26:42.978111 9244 tablet_replica.cc:439] T c54e475da85948da9fca4001755162b4 P b016084d0efc4659b75da627a2c14a6e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } } }
I20260808 22:26:42.978236 9244 tablet_replica.cc:442] T c54e475da85948da9fca4001755162b4 P b016084d0efc4659b75da627a2c14a6e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:26:42.978367 9228 ts_tablet_manager.cc:1428] T c54e475da85948da9fca4001755162b4 P 38ebf0799f9044a688c56196e95a4f8f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:26:42.978544 9229 ts_tablet_manager.cc:1428] T c54e475da85948da9fca4001755162b4 P b016084d0efc4659b75da627a2c14a6e: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 22:26:42.978565 9243 tablet_replica.cc:439] T c54e475da85948da9fca4001755162b4 P 38ebf0799f9044a688c56196e95a4f8f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } } }
I20260808 22:26:42.978694 9243 tablet_replica.cc:442] T c54e475da85948da9fca4001755162b4 P 38ebf0799f9044a688c56196e95a4f8f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:26:42.979291 9227 ts_tablet_manager.cc:1428] T c54e475da85948da9fca4001755162b4 P 362b664198bb4b25aacfd84c6f0d5119: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:26:43.032593 9244 raft_consensus.cc:482] T c54e475da85948da9fca4001755162b4 P b016084d0efc4659b75da627a2c14a6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:26:43.032727 9244 raft_consensus.cc:504] T c54e475da85948da9fca4001755162b4 P b016084d0efc4659b75da627a2c14a6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } }
I20260808 22:26:43.033891 9244 leader_election.cc:288] T c54e475da85948da9fca4001755162b4 P b016084d0efc4659b75da627a2c14a6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 362b664198bb4b25aacfd84c6f0d5119 (127.5.11.193:44745), 38ebf0799f9044a688c56196e95a4f8f (127.5.11.194:36057)
I20260808 22:26:43.038034 8988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:43.034093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:44745 (local address 127.5.11.195:36367)
0808 22:26:43.034277 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:43.034292 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:43.034397 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:43.034670 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:43.034673 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:43.034693 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:26:43.034835 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:43.034842 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:43.035652 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:43.035656 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:43.036867 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:43.036877 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:43.037549 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:43.037553 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:43.037664 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:43.037698 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 22:26:43.037793 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 22:26:43.037861 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":18}
I20260808 22:26:43.038072 9221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:43.034174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:36367 (local address 127.5.11.193:44745)
0808 22:26:43.034299 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:43.034305 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:43.034424 (+ 119us) server_negotiation.cc:407] Connection header received
0808 22:26:43.034494 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:43.034500 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:43.034558 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:43.034639 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:26:43.034957 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:43.035538 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:43.036989 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:43.037413 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:43.037565 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:43.037674 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:43.037840 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 22:26:43.037917 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 22:26:43.038348 9224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:43.034273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:57353 (local address 127.5.11.194:36057)
0808 22:26:43.034413 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:43.034419 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:43.034556 (+ 137us) server_negotiation.cc:407] Connection header received
0808 22:26:43.034680 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:43.034683 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:43.034733 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:43.034819 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:26:43.035137 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:43.035662 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:43.037087 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:43.037548 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:43.037817 (+ 269us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:43.037866 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:43.038113 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 22:26:43.038203 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 22:26:43.038561 9263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:43.034119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:36057 (local address 127.5.11.195:57353)
0808 22:26:43.034456 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:43.034470 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:43.034570 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:43.034843 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:43.034845 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:43.034866 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:26:43.035020 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:43.035026 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:43.035796 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:43.035800 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:43.036951 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:43.036964 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:43.037834 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:43.037838 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:43.037939 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:43.037962 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 22:26:43.038041 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 22:26:43.038418 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"mutex_wait_us":57,"thread_start_us":126,"threads_started":1}
I20260808 22:26:43.038529 8592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c54e475da85948da9fca4001755162b4" candidate_uuid: "b016084d0efc4659b75da627a2c14a6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "362b664198bb4b25aacfd84c6f0d5119" is_pre_election: true
I20260808 22:26:43.038673 8592 raft_consensus.cc:2454] T c54e475da85948da9fca4001755162b4 P 362b664198bb4b25aacfd84c6f0d5119 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b016084d0efc4659b75da627a2c14a6e in term 0.
I20260808 22:26:43.039036 8975 leader_election.cc:302] T c54e475da85948da9fca4001755162b4 P b016084d0efc4659b75da627a2c14a6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 362b664198bb4b25aacfd84c6f0d5119, b016084d0efc4659b75da627a2c14a6e; no voters:
I20260808 22:26:43.039148 8850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c54e475da85948da9fca4001755162b4" candidate_uuid: "b016084d0efc4659b75da627a2c14a6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38ebf0799f9044a688c56196e95a4f8f" is_pre_election: true
I20260808 22:26:43.039348 9244 raft_consensus.cc:2781] T c54e475da85948da9fca4001755162b4 P b016084d0efc4659b75da627a2c14a6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:26:43.039371 8850 raft_consensus.cc:2454] T c54e475da85948da9fca4001755162b4 P 38ebf0799f9044a688c56196e95a4f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b016084d0efc4659b75da627a2c14a6e in term 0.
I20260808 22:26:43.039438 9244 raft_consensus.cc:482] T c54e475da85948da9fca4001755162b4 P b016084d0efc4659b75da627a2c14a6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:26:43.039510 9244 raft_consensus.cc:3037] T c54e475da85948da9fca4001755162b4 P b016084d0efc4659b75da627a2c14a6e [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:43.040723 9244 raft_consensus.cc:504] T c54e475da85948da9fca4001755162b4 P b016084d0efc4659b75da627a2c14a6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } }
I20260808 22:26:43.041090 9244 leader_election.cc:288] T c54e475da85948da9fca4001755162b4 P b016084d0efc4659b75da627a2c14a6e [CANDIDATE]: Term 1 election: Requested vote from peers 362b664198bb4b25aacfd84c6f0d5119 (127.5.11.193:44745), 38ebf0799f9044a688c56196e95a4f8f (127.5.11.194:36057)
I20260808 22:26:43.041373 8592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c54e475da85948da9fca4001755162b4" candidate_uuid: "b016084d0efc4659b75da627a2c14a6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "362b664198bb4b25aacfd84c6f0d5119"
I20260808 22:26:43.041395 8850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c54e475da85948da9fca4001755162b4" candidate_uuid: "b016084d0efc4659b75da627a2c14a6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38ebf0799f9044a688c56196e95a4f8f"
I20260808 22:26:43.041492 8850 raft_consensus.cc:3037] T c54e475da85948da9fca4001755162b4 P 38ebf0799f9044a688c56196e95a4f8f [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:43.041492 8592 raft_consensus.cc:3037] T c54e475da85948da9fca4001755162b4 P 362b664198bb4b25aacfd84c6f0d5119 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:43.042641 8850 raft_consensus.cc:2454] T c54e475da85948da9fca4001755162b4 P 38ebf0799f9044a688c56196e95a4f8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b016084d0efc4659b75da627a2c14a6e in term 1.
I20260808 22:26:43.042641 8592 raft_consensus.cc:2454] T c54e475da85948da9fca4001755162b4 P 362b664198bb4b25aacfd84c6f0d5119 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b016084d0efc4659b75da627a2c14a6e in term 1.
I20260808 22:26:43.042919 8975 leader_election.cc:302] T c54e475da85948da9fca4001755162b4 P b016084d0efc4659b75da627a2c14a6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 362b664198bb4b25aacfd84c6f0d5119, b016084d0efc4659b75da627a2c14a6e; no voters:
I20260808 22:26:43.043115 9244 raft_consensus.cc:2781] T c54e475da85948da9fca4001755162b4 P b016084d0efc4659b75da627a2c14a6e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:26:43.043340 9244 raft_consensus.cc:686] T c54e475da85948da9fca4001755162b4 P b016084d0efc4659b75da627a2c14a6e [term 1 LEADER]: Becoming Leader. State: Replica: b016084d0efc4659b75da627a2c14a6e, State: Running, Role: LEADER
I20260808 22:26:43.043686 9244 consensus_queue.cc:237] T c54e475da85948da9fca4001755162b4 P b016084d0efc4659b75da627a2c14a6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } }
I20260808 22:26:43.046017 9266 tablet_replica.cc:439] T c54e475da85948da9fca4001755162b4 P b016084d0efc4659b75da627a2c14a6e: TxnStatusTablet state changed. Reason: New leader b016084d0efc4659b75da627a2c14a6e. Latest consensus state: current_term: 1 leader_uuid: "b016084d0efc4659b75da627a2c14a6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } } }
I20260808 22:26:43.046133 9266 tablet_replica.cc:442] T c54e475da85948da9fca4001755162b4 P b016084d0efc4659b75da627a2c14a6e: This TxnStatusTablet replica's current role is: LEADER
I20260808 22:26:43.046453 9270 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 22:26:43.046658 9270 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 22:26:43.046974 8305 catalog_manager.cc:5495] T c54e475da85948da9fca4001755162b4 P b016084d0efc4659b75da627a2c14a6e reported cstate change: term changed from 0 to 1, leader changed from <none> to b016084d0efc4659b75da627a2c14a6e (127.5.11.195). New cstate: current_term: 1 leader_uuid: "b016084d0efc4659b75da627a2c14a6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } health_report { overall_health: HEALTHY } } }
I20260808 22:26:43.061306 9217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:43.057505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41968 (local address 127.5.11.195:41455)
0808 22:26:43.057628 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:43.057631 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:43.057642 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:26:43.057830 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:43.057832 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:43.057894 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:43.057972 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:26:43.058239 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:43.058730 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:43.059976 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:43.060204 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:43.060280 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:43.060613 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:26:43.060975 (+ 362us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:26:43.061058 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:43.061175 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 22:26:43.061217 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 22:26:43.061376 9255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:43.057404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:41455 (local address 127.0.0.1:41968)
0808 22:26:43.057588 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:43.057604 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:43.057700 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:43.058015 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:43.058018 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:43.058029 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:26:43.058134 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:43.058141 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:43.058833 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:43.058837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:43.059866 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:43.059878 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:43.060310 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:43.060314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:43.060407 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:43.060463 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:26:43.061067 (+ 604us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:26:43.061082 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 22:26:43.061156 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 22:26:43.061243 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 22:26:43.066082 8850 raft_consensus.cc:1261] T c54e475da85948da9fca4001755162b4 P 38ebf0799f9044a688c56196e95a4f8f [term 1 FOLLOWER]: Refusing update from remote peer b016084d0efc4659b75da627a2c14a6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:26:43.066139 8592 raft_consensus.cc:1261] T c54e475da85948da9fca4001755162b4 P 362b664198bb4b25aacfd84c6f0d5119 [term 1 FOLLOWER]: Refusing update from remote peer b016084d0efc4659b75da627a2c14a6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:26:43.066699 9244 consensus_queue.cc:1035] T c54e475da85948da9fca4001755162b4 P b016084d0efc4659b75da627a2c14a6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:26:43.066965 9266 consensus_queue.cc:1035] T c54e475da85948da9fca4001755162b4 P b016084d0efc4659b75da627a2c14a6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:26:43.070392 9243 tablet_replica.cc:439] T c54e475da85948da9fca4001755162b4 P 38ebf0799f9044a688c56196e95a4f8f: TxnStatusTablet state changed. Reason: New leader b016084d0efc4659b75da627a2c14a6e. Latest consensus state: current_term: 1 leader_uuid: "b016084d0efc4659b75da627a2c14a6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } } }
I20260808 22:26:43.070544 9243 tablet_replica.cc:442] T c54e475da85948da9fca4001755162b4 P 38ebf0799f9044a688c56196e95a4f8f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:26:43.070804 9242 tablet_replica.cc:439] T c54e475da85948da9fca4001755162b4 P 362b664198bb4b25aacfd84c6f0d5119: TxnStatusTablet state changed. Reason: New leader b016084d0efc4659b75da627a2c14a6e. Latest consensus state: current_term: 1 leader_uuid: "b016084d0efc4659b75da627a2c14a6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } } }
I20260808 22:26:43.070902 9242 tablet_replica.cc:442] T c54e475da85948da9fca4001755162b4 P 362b664198bb4b25aacfd84c6f0d5119: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:26:43.074260 9266 tablet_replica.cc:439] T c54e475da85948da9fca4001755162b4 P b016084d0efc4659b75da627a2c14a6e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b016084d0efc4659b75da627a2c14a6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } } }
I20260808 22:26:43.074363 9266 tablet_replica.cc:442] T c54e475da85948da9fca4001755162b4 P b016084d0efc4659b75da627a2c14a6e: This TxnStatusTablet replica's current role is: LEADER
I20260808 22:26:43.074880 9278 mvcc.cc:205] Tried to move back new op lower bound from 7316389900552286208 to 7316389900469661696. Current Snapshot: MvccSnapshot[applied={T|T < 7316389900552286208}]
I20260808 22:26:43.075388 9243 tablet_replica.cc:439] T c54e475da85948da9fca4001755162b4 P 38ebf0799f9044a688c56196e95a4f8f: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b016084d0efc4659b75da627a2c14a6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } } }
I20260808 22:26:43.075527 9243 tablet_replica.cc:442] T c54e475da85948da9fca4001755162b4 P 38ebf0799f9044a688c56196e95a4f8f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:26:43.075516 9266 tablet_replica.cc:439] T c54e475da85948da9fca4001755162b4 P b016084d0efc4659b75da627a2c14a6e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b016084d0efc4659b75da627a2c14a6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } } }
I20260808 22:26:43.075599 9266 tablet_replica.cc:442] T c54e475da85948da9fca4001755162b4 P b016084d0efc4659b75da627a2c14a6e: This TxnStatusTablet replica's current role is: LEADER
I20260808 22:26:43.076769 9242 tablet_replica.cc:439] T c54e475da85948da9fca4001755162b4 P 362b664198bb4b25aacfd84c6f0d5119: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b016084d0efc4659b75da627a2c14a6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } } }
I20260808 22:26:43.076871 9242 tablet_replica.cc:442] T c54e475da85948da9fca4001755162b4 P 362b664198bb4b25aacfd84c6f0d5119: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:26:43.087927 9224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:43.083917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50602 (local address 127.5.11.194:36057)
0808 22:26:43.084304 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:43.084308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:43.084320 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:26:43.084384 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:43.084387 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:43.084441 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:43.084548 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:26:43.084789 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:43.085562 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:43.086374 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:43.086729 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:43.086846 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:43.087096 (+ 250us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:26:43.087559 (+ 463us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:26:43.087675 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:43.087730 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 22:26:43.087802 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292}
I20260808 22:26:43.098008 9217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:43.094666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:36895 (local address 127.5.11.195:41455)
0808 22:26:43.094800 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:43.094805 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:43.094956 (+ 151us) server_negotiation.cc:407] Connection header received
0808 22:26:43.095115 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:43.095118 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:43.095166 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:43.095241 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:26:43.095499 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:43.096056 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:43.096932 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:43.097144 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:43.097258 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:43.097586 (+ 328us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:26:43.097646 (+ 60us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:26:43.097719 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:43.097838 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 22:26:43.097884 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 22:26:43.098055 9298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:43.094561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:41455 (local address 127.5.11.194:36895)
0808 22:26:43.094874 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:43.094887 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:43.095007 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:43.095252 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:43.095254 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:43.095262 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:26:43.095379 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:43.095384 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:43.096160 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:43.096167 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:43.096799 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:43.096808 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:43.097273 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:43.097277 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:43.097371 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:43.097410 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:26:43.097738 (+ 328us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:26:43.097745 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 22:26:43.097825 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 22:26:43.097906 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":99,"threads_started":1}
I20260808 22:26:43.105254 9111 raft_consensus.cc:1261] T debe1d8bd94c49cea47f4ffff243dac4 P b016084d0efc4659b75da627a2c14a6e [term 1 FOLLOWER]: Refusing update from remote peer 38ebf0799f9044a688c56196e95a4f8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:26:43.105607 8592 raft_consensus.cc:1261] T debe1d8bd94c49cea47f4ffff243dac4 P 362b664198bb4b25aacfd84c6f0d5119 [term 1 FOLLOWER]: Refusing update from remote peer 38ebf0799f9044a688c56196e95a4f8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:26:43.105907 9247 consensus_queue.cc:1035] T debe1d8bd94c49cea47f4ffff243dac4 P 38ebf0799f9044a688c56196e95a4f8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:26:43.106066 9243 consensus_queue.cc:1035] T debe1d8bd94c49cea47f4ffff243dac4 P 38ebf0799f9044a688c56196e95a4f8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:26:43.111157 9274 mvcc.cc:205] Tried to move back new op lower bound from 7316389900713766912 to 7316389899975700480. Current Snapshot: MvccSnapshot[applied={T|T < 7316389900713766912}]
I20260808 22:26:43.111347 9275 mvcc.cc:205] Tried to move back new op lower bound from 7316389900713766912 to 7316389899975700480. Current Snapshot: MvccSnapshot[applied={T|T < 7316389900713766912}]
I20260808 22:26:43.129583 9195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:43.125746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:36057 (local address 127.5.11.195:55815)
0808 22:26:43.125912 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:43.125926 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:43.125994 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:43.126269 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:43.126271 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:43.126276 (+ 5us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:26:43.126371 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:43.126375 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:43.127402 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:43.127405 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:43.128350 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:43.128372 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:43.128953 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:43.128956 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:43.129028 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:43.129073 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:26:43.129366 (+ 293us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:26:43.129372 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 22:26:43.129434 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 22:26:43.129490 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260808 22:26:43.129617 9224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:43.125969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:55815 (local address 127.5.11.194:36057)
0808 22:26:43.126067 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:43.126073 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:43.126085 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:26:43.126128 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:43.126131 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:43.126176 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:43.126258 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:26:43.126470 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:43.127279 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:43.128494 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:43.128839 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:43.128946 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:43.129196 (+ 250us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:26:43.129277 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:26:43.129351 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:43.129445 (+ 94us) server_negotiation.cc:299] Negotiation successful
0808 22:26:43.129500 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260808 22:26:43.156599 8305 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:26:43.156792 8305 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:26:43.159601 8305 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 08a9e7267a0246319859b8084419fecb: Sending DeleteTablet for 3 replicas of tablet debe1d8bd94c49cea47f4ffff243dac4
I20260808 22:26:43.160346 9069 tablet_service.cc:1514] Processing DeleteTablet for tablet debe1d8bd94c49cea47f4ffff243dac4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:26:43 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:41954
I20260808 22:26:43.160326 8559 tablet_service.cc:1514] Processing DeleteTablet for tablet debe1d8bd94c49cea47f4ffff243dac4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:26:43 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:58640
I20260808 22:26:43.160809 8803 tablet_service.cc:1514] Processing DeleteTablet for tablet debe1d8bd94c49cea47f4ffff243dac4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:26:43 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:50598
I20260808 22:26:43.161211 8303 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45120:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 22:26:43.161597 9314 tablet_replica.cc:330] T debe1d8bd94c49cea47f4ffff243dac4 P 362b664198bb4b25aacfd84c6f0d5119: stopping tablet replica
W20260808 22:26:43.161639 8303 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:26:43.162022 9314 raft_consensus.cc:2229] T debe1d8bd94c49cea47f4ffff243dac4 P 362b664198bb4b25aacfd84c6f0d5119 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:26:43.162309 9314 raft_consensus.cc:2258] T debe1d8bd94c49cea47f4ffff243dac4 P 362b664198bb4b25aacfd84c6f0d5119 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:26:43.162772 9313 tablet_replica.cc:330] T debe1d8bd94c49cea47f4ffff243dac4 P b016084d0efc4659b75da627a2c14a6e: stopping tablet replica
I20260808 22:26:43.163110 9313 raft_consensus.cc:2229] T debe1d8bd94c49cea47f4ffff243dac4 P b016084d0efc4659b75da627a2c14a6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:26:43.163316 9313 raft_consensus.cc:2258] T debe1d8bd94c49cea47f4ffff243dac4 P b016084d0efc4659b75da627a2c14a6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:26:43.164877 9313 ts_tablet_manager.cc:1905] T debe1d8bd94c49cea47f4ffff243dac4 P b016084d0efc4659b75da627a2c14a6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:26:43.167316 9069 tablet_service.cc:1467] Processing CreateTablet for tablet e77ab6b393bd45b4bdcef8cd29e83856 (DEFAULT_TABLE table=test-table [id=486d9cd6fb444986af7f3f00b8554180]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:26:43.167330 8559 tablet_service.cc:1467] Processing CreateTablet for tablet e77ab6b393bd45b4bdcef8cd29e83856 (DEFAULT_TABLE table=test-table [id=486d9cd6fb444986af7f3f00b8554180]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:26:43.167588 8559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e77ab6b393bd45b4bdcef8cd29e83856. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:43.167594 9069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e77ab6b393bd45b4bdcef8cd29e83856. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:43.169188 9314 ts_tablet_manager.cc:1905] T debe1d8bd94c49cea47f4ffff243dac4 P 362b664198bb4b25aacfd84c6f0d5119: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:26:43.170090 9229 tablet_bootstrap.cc:492] T e77ab6b393bd45b4bdcef8cd29e83856 P b016084d0efc4659b75da627a2c14a6e: Bootstrap starting.
I20260808 22:26:43.170100 8803 tablet_service.cc:1467] Processing CreateTablet for tablet e77ab6b393bd45b4bdcef8cd29e83856 (DEFAULT_TABLE table=test-table [id=486d9cd6fb444986af7f3f00b8554180]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:26:43.170346 8803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e77ab6b393bd45b4bdcef8cd29e83856. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:43.171046 9229 tablet_bootstrap.cc:654] T e77ab6b393bd45b4bdcef8cd29e83856 P b016084d0efc4659b75da627a2c14a6e: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:43.171979 9315 tablet_replica.cc:330] T debe1d8bd94c49cea47f4ffff243dac4 P 38ebf0799f9044a688c56196e95a4f8f: stopping tablet replica
I20260808 22:26:43.172286 9315 raft_consensus.cc:2229] T debe1d8bd94c49cea47f4ffff243dac4 P 38ebf0799f9044a688c56196e95a4f8f [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:26:43.172657 9315 raft_consensus.cc:2258] T debe1d8bd94c49cea47f4ffff243dac4 P 38ebf0799f9044a688c56196e95a4f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:26:43.172729 9227 tablet_bootstrap.cc:492] T e77ab6b393bd45b4bdcef8cd29e83856 P 362b664198bb4b25aacfd84c6f0d5119: Bootstrap starting.
I20260808 22:26:43.173028 9228 tablet_bootstrap.cc:492] T e77ab6b393bd45b4bdcef8cd29e83856 P 38ebf0799f9044a688c56196e95a4f8f: Bootstrap starting.
I20260808 22:26:43.173816 9314 ts_tablet_manager.cc:1918] T debe1d8bd94c49cea47f4ffff243dac4 P 362b664198bb4b25aacfd84c6f0d5119: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:26:43.173906 9314 log.cc:1196] T debe1d8bd94c49cea47f4ffff243dac4 P 362b664198bb4b25aacfd84c6f0d5119: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-0/wal/wals/debe1d8bd94c49cea47f4ffff243dac4
I20260808 22:26:43.173902 9228 tablet_bootstrap.cc:654] T e77ab6b393bd45b4bdcef8cd29e83856 P 38ebf0799f9044a688c56196e95a4f8f: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:43.174247 9314 ts_tablet_manager.cc:1939] T debe1d8bd94c49cea47f4ffff243dac4 P 362b664198bb4b25aacfd84c6f0d5119: Deleting consensus metadata
I20260808 22:26:43.175920 9227 tablet_bootstrap.cc:654] T e77ab6b393bd45b4bdcef8cd29e83856 P 362b664198bb4b25aacfd84c6f0d5119: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:43.176281 8274 catalog_manager.cc:4841] TS 362b664198bb4b25aacfd84c6f0d5119 (127.5.11.193:44745): tablet debe1d8bd94c49cea47f4ffff243dac4 (table test-table [id=5befd864b1c14a29a1e8be0505f29dd2]) successfully deleted
I20260808 22:26:43.177587 9315 ts_tablet_manager.cc:1905] T debe1d8bd94c49cea47f4ffff243dac4 P 38ebf0799f9044a688c56196e95a4f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:26:43.177980 9313 ts_tablet_manager.cc:1918] T debe1d8bd94c49cea47f4ffff243dac4 P b016084d0efc4659b75da627a2c14a6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:26:43.178136 9313 log.cc:1196] T debe1d8bd94c49cea47f4ffff243dac4 P b016084d0efc4659b75da627a2c14a6e: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-2/wal/wals/debe1d8bd94c49cea47f4ffff243dac4
I20260808 22:26:43.178555 9313 ts_tablet_manager.cc:1939] T debe1d8bd94c49cea47f4ffff243dac4 P b016084d0efc4659b75da627a2c14a6e: Deleting consensus metadata
I20260808 22:26:43.179775 8277 catalog_manager.cc:4841] TS b016084d0efc4659b75da627a2c14a6e (127.5.11.195:41455): tablet debe1d8bd94c49cea47f4ffff243dac4 (table test-table [id=5befd864b1c14a29a1e8be0505f29dd2]) successfully deleted
I20260808 22:26:43.180671 9228 tablet_bootstrap.cc:492] T e77ab6b393bd45b4bdcef8cd29e83856 P 38ebf0799f9044a688c56196e95a4f8f: No bootstrap required, opened a new log
I20260808 22:26:43.180748 9228 ts_tablet_manager.cc:1397] T e77ab6b393bd45b4bdcef8cd29e83856 P 38ebf0799f9044a688c56196e95a4f8f: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260808 22:26:43.181151 9228 raft_consensus.cc:348] T e77ab6b393bd45b4bdcef8cd29e83856 P 38ebf0799f9044a688c56196e95a4f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } }
I20260808 22:26:43.181253 9228 raft_consensus.cc:374] T e77ab6b393bd45b4bdcef8cd29e83856 P 38ebf0799f9044a688c56196e95a4f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:43.181283 9228 raft_consensus.cc:729] T e77ab6b393bd45b4bdcef8cd29e83856 P 38ebf0799f9044a688c56196e95a4f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38ebf0799f9044a688c56196e95a4f8f, State: Initialized, Role: FOLLOWER
I20260808 22:26:43.181396 9228 consensus_queue.cc:260] T e77ab6b393bd45b4bdcef8cd29e83856 P 38ebf0799f9044a688c56196e95a4f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } }
I20260808 22:26:43.181633 9228 ts_tablet_manager.cc:1428] T e77ab6b393bd45b4bdcef8cd29e83856 P 38ebf0799f9044a688c56196e95a4f8f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:26:43.182598 9229 tablet_bootstrap.cc:492] T e77ab6b393bd45b4bdcef8cd29e83856 P b016084d0efc4659b75da627a2c14a6e: No bootstrap required, opened a new log
I20260808 22:26:43.182701 9229 ts_tablet_manager.cc:1397] T e77ab6b393bd45b4bdcef8cd29e83856 P b016084d0efc4659b75da627a2c14a6e: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.001s
I20260808 22:26:43.182971 9227 tablet_bootstrap.cc:492] T e77ab6b393bd45b4bdcef8cd29e83856 P 362b664198bb4b25aacfd84c6f0d5119: No bootstrap required, opened a new log
I20260808 22:26:43.183070 9227 ts_tablet_manager.cc:1397] T e77ab6b393bd45b4bdcef8cd29e83856 P 362b664198bb4b25aacfd84c6f0d5119: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s
I20260808 22:26:43.183168 9229 raft_consensus.cc:348] T e77ab6b393bd45b4bdcef8cd29e83856 P b016084d0efc4659b75da627a2c14a6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } }
I20260808 22:26:43.183322 9229 raft_consensus.cc:374] T e77ab6b393bd45b4bdcef8cd29e83856 P b016084d0efc4659b75da627a2c14a6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:43.183357 9229 raft_consensus.cc:729] T e77ab6b393bd45b4bdcef8cd29e83856 P b016084d0efc4659b75da627a2c14a6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b016084d0efc4659b75da627a2c14a6e, State: Initialized, Role: FOLLOWER
I20260808 22:26:43.183481 9229 consensus_queue.cc:260] T e77ab6b393bd45b4bdcef8cd29e83856 P b016084d0efc4659b75da627a2c14a6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } }
I20260808 22:26:43.183555 9227 raft_consensus.cc:348] T e77ab6b393bd45b4bdcef8cd29e83856 P 362b664198bb4b25aacfd84c6f0d5119 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } }
I20260808 22:26:43.183669 9227 raft_consensus.cc:374] T e77ab6b393bd45b4bdcef8cd29e83856 P 362b664198bb4b25aacfd84c6f0d5119 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:43.183712 9227 raft_consensus.cc:729] T e77ab6b393bd45b4bdcef8cd29e83856 P 362b664198bb4b25aacfd84c6f0d5119 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 362b664198bb4b25aacfd84c6f0d5119, State: Initialized, Role: FOLLOWER
I20260808 22:26:43.183722 9229 ts_tablet_manager.cc:1428] T e77ab6b393bd45b4bdcef8cd29e83856 P b016084d0efc4659b75da627a2c14a6e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:26:43.183887 9227 consensus_queue.cc:260] T e77ab6b393bd45b4bdcef8cd29e83856 P 362b664198bb4b25aacfd84c6f0d5119 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } }
I20260808 22:26:43.184188 9227 ts_tablet_manager.cc:1428] T e77ab6b393bd45b4bdcef8cd29e83856 P 362b664198bb4b25aacfd84c6f0d5119: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 22:26:43.184473 9315 ts_tablet_manager.cc:1918] T debe1d8bd94c49cea47f4ffff243dac4 P 38ebf0799f9044a688c56196e95a4f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:26:43.184577 9315 log.cc:1196] T debe1d8bd94c49cea47f4ffff243dac4 P 38ebf0799f9044a688c56196e95a4f8f: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-1/wal/wals/debe1d8bd94c49cea47f4ffff243dac4
I20260808 22:26:43.184919 9315 ts_tablet_manager.cc:1939] T debe1d8bd94c49cea47f4ffff243dac4 P 38ebf0799f9044a688c56196e95a4f8f: Deleting consensus metadata
I20260808 22:26:43.186923 8281 catalog_manager.cc:4841] TS 38ebf0799f9044a688c56196e95a4f8f (127.5.11.194:36057): tablet debe1d8bd94c49cea47f4ffff243dac4 (table test-table [id=5befd864b1c14a29a1e8be0505f29dd2]) successfully deleted
I20260808 22:26:43.208567 9310 raft_consensus.cc:482] T e77ab6b393bd45b4bdcef8cd29e83856 P 38ebf0799f9044a688c56196e95a4f8f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:26:43.208693 9310 raft_consensus.cc:504] T e77ab6b393bd45b4bdcef8cd29e83856 P 38ebf0799f9044a688c56196e95a4f8f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } }
I20260808 22:26:43.209121 9310 leader_election.cc:288] T e77ab6b393bd45b4bdcef8cd29e83856 P 38ebf0799f9044a688c56196e95a4f8f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 362b664198bb4b25aacfd84c6f0d5119 (127.5.11.193:44745), b016084d0efc4659b75da627a2c14a6e (127.5.11.195:41455)
I20260808 22:26:43.209496 8592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e77ab6b393bd45b4bdcef8cd29e83856" candidate_uuid: "38ebf0799f9044a688c56196e95a4f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "362b664198bb4b25aacfd84c6f0d5119" is_pre_election: true
I20260808 22:26:43.209640 8592 raft_consensus.cc:2454] T e77ab6b393bd45b4bdcef8cd29e83856 P 362b664198bb4b25aacfd84c6f0d5119 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38ebf0799f9044a688c56196e95a4f8f in term 0.
I20260808 22:26:43.210281 9110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e77ab6b393bd45b4bdcef8cd29e83856" candidate_uuid: "38ebf0799f9044a688c56196e95a4f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b016084d0efc4659b75da627a2c14a6e" is_pre_election: true
I20260808 22:26:43.210439 9110 raft_consensus.cc:2454] T e77ab6b393bd45b4bdcef8cd29e83856 P b016084d0efc4659b75da627a2c14a6e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38ebf0799f9044a688c56196e95a4f8f in term 0.
I20260808 22:26:43.210667 8724 leader_election.cc:302] T e77ab6b393bd45b4bdcef8cd29e83856 P 38ebf0799f9044a688c56196e95a4f8f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 362b664198bb4b25aacfd84c6f0d5119, 38ebf0799f9044a688c56196e95a4f8f; no voters:
I20260808 22:26:43.211217 9310 raft_consensus.cc:2781] T e77ab6b393bd45b4bdcef8cd29e83856 P 38ebf0799f9044a688c56196e95a4f8f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:26:43.211279 9310 raft_consensus.cc:482] T e77ab6b393bd45b4bdcef8cd29e83856 P 38ebf0799f9044a688c56196e95a4f8f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:26:43.211318 9310 raft_consensus.cc:3037] T e77ab6b393bd45b4bdcef8cd29e83856 P 38ebf0799f9044a688c56196e95a4f8f [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:43.212563 9310 raft_consensus.cc:504] T e77ab6b393bd45b4bdcef8cd29e83856 P 38ebf0799f9044a688c56196e95a4f8f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } }
I20260808 22:26:43.212886 9310 leader_election.cc:288] T e77ab6b393bd45b4bdcef8cd29e83856 P 38ebf0799f9044a688c56196e95a4f8f [CANDIDATE]: Term 1 election: Requested vote from peers 362b664198bb4b25aacfd84c6f0d5119 (127.5.11.193:44745), b016084d0efc4659b75da627a2c14a6e (127.5.11.195:41455)
I20260808 22:26:43.213280 9110 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e77ab6b393bd45b4bdcef8cd29e83856" candidate_uuid: "38ebf0799f9044a688c56196e95a4f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b016084d0efc4659b75da627a2c14a6e"
I20260808 22:26:43.213393 9110 raft_consensus.cc:3037] T e77ab6b393bd45b4bdcef8cd29e83856 P b016084d0efc4659b75da627a2c14a6e [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:43.215081 9110 raft_consensus.cc:2454] T e77ab6b393bd45b4bdcef8cd29e83856 P b016084d0efc4659b75da627a2c14a6e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38ebf0799f9044a688c56196e95a4f8f in term 1.
I20260808 22:26:43.215641 8592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e77ab6b393bd45b4bdcef8cd29e83856" candidate_uuid: "38ebf0799f9044a688c56196e95a4f8f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "362b664198bb4b25aacfd84c6f0d5119"
I20260808 22:26:43.215770 8592 raft_consensus.cc:3037] T e77ab6b393bd45b4bdcef8cd29e83856 P 362b664198bb4b25aacfd84c6f0d5119 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:43.216857 8592 raft_consensus.cc:2454] T e77ab6b393bd45b4bdcef8cd29e83856 P 362b664198bb4b25aacfd84c6f0d5119 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38ebf0799f9044a688c56196e95a4f8f in term 1.
I20260808 22:26:43.217854 8725 leader_election.cc:302] T e77ab6b393bd45b4bdcef8cd29e83856 P 38ebf0799f9044a688c56196e95a4f8f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38ebf0799f9044a688c56196e95a4f8f, b016084d0efc4659b75da627a2c14a6e; no voters:
I20260808 22:26:43.218330 9310 raft_consensus.cc:2781] T e77ab6b393bd45b4bdcef8cd29e83856 P 38ebf0799f9044a688c56196e95a4f8f [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:26:43.218443 9310 raft_consensus.cc:686] T e77ab6b393bd45b4bdcef8cd29e83856 P 38ebf0799f9044a688c56196e95a4f8f [term 1 LEADER]: Becoming Leader. State: Replica: 38ebf0799f9044a688c56196e95a4f8f, State: Running, Role: LEADER
I20260808 22:26:43.218585 9310 consensus_queue.cc:237] T e77ab6b393bd45b4bdcef8cd29e83856 P 38ebf0799f9044a688c56196e95a4f8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } }
I20260808 22:26:43.220520 8303 catalog_manager.cc:5495] T e77ab6b393bd45b4bdcef8cd29e83856 P 38ebf0799f9044a688c56196e95a4f8f reported cstate change: term changed from 0 to 1, leader changed from <none> to 38ebf0799f9044a688c56196e95a4f8f (127.5.11.194). New cstate: current_term: 1 leader_uuid: "38ebf0799f9044a688c56196e95a4f8f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } health_report { overall_health: UNKNOWN } } }
I20260808 22:26:43.242688 8592 raft_consensus.cc:1261] T e77ab6b393bd45b4bdcef8cd29e83856 P 362b664198bb4b25aacfd84c6f0d5119 [term 1 FOLLOWER]: Refusing update from remote peer 38ebf0799f9044a688c56196e95a4f8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:26:43.242867 9110 raft_consensus.cc:1261] T e77ab6b393bd45b4bdcef8cd29e83856 P b016084d0efc4659b75da627a2c14a6e [term 1 FOLLOWER]: Refusing update from remote peer 38ebf0799f9044a688c56196e95a4f8f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:26:43.243357 9310 consensus_queue.cc:1035] T e77ab6b393bd45b4bdcef8cd29e83856 P 38ebf0799f9044a688c56196e95a4f8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:26:43.243567 9305 consensus_queue.cc:1035] T e77ab6b393bd45b4bdcef8cd29e83856 P 38ebf0799f9044a688c56196e95a4f8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 22:26:43.259407 9278 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 22:26:44.810027 8305 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:26:44.810213 8305 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45120:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:26:44.812469 8305 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 08a9e7267a0246319859b8084419fecb: Sending DeleteTablet for 3 replicas of tablet e77ab6b393bd45b4bdcef8cd29e83856
I20260808 22:26:44.813005 8559 tablet_service.cc:1514] Processing DeleteTablet for tablet e77ab6b393bd45b4bdcef8cd29e83856 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:26:44 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:58640
I20260808 22:26:44.813081 8803 tablet_service.cc:1514] Processing DeleteTablet for tablet e77ab6b393bd45b4bdcef8cd29e83856 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:26:44 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:50598
I20260808 22:26:44.813108 9069 tablet_service.cc:1514] Processing DeleteTablet for tablet e77ab6b393bd45b4bdcef8cd29e83856 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:26:44 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:41954
I20260808 22:26:44.813386 9372 tablet_replica.cc:330] T e77ab6b393bd45b4bdcef8cd29e83856 P 38ebf0799f9044a688c56196e95a4f8f: stopping tablet replica
I20260808 22:26:44.813429 9373 tablet_replica.cc:330] T e77ab6b393bd45b4bdcef8cd29e83856 P b016084d0efc4659b75da627a2c14a6e: stopping tablet replica
I20260808 22:26:44.813558 9373 raft_consensus.cc:2229] T e77ab6b393bd45b4bdcef8cd29e83856 P b016084d0efc4659b75da627a2c14a6e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:26:44.813580 9372 raft_consensus.cc:2229] T e77ab6b393bd45b4bdcef8cd29e83856 P 38ebf0799f9044a688c56196e95a4f8f [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:26:44.813678 9373 raft_consensus.cc:2258] T e77ab6b393bd45b4bdcef8cd29e83856 P b016084d0efc4659b75da627a2c14a6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:26:44.813803 9372 raft_consensus.cc:2258] T e77ab6b393bd45b4bdcef8cd29e83856 P 38ebf0799f9044a688c56196e95a4f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:26:44.814390 9372 ts_tablet_manager.cc:1905] T e77ab6b393bd45b4bdcef8cd29e83856 P 38ebf0799f9044a688c56196e95a4f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:26:44.814438 9373 ts_tablet_manager.cc:1905] T e77ab6b393bd45b4bdcef8cd29e83856 P b016084d0efc4659b75da627a2c14a6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:26:44.816885 9372 ts_tablet_manager.cc:1918] T e77ab6b393bd45b4bdcef8cd29e83856 P 38ebf0799f9044a688c56196e95a4f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 22:26:44.816983 9372 log.cc:1196] T e77ab6b393bd45b4bdcef8cd29e83856 P 38ebf0799f9044a688c56196e95a4f8f: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-1/wal/wals/e77ab6b393bd45b4bdcef8cd29e83856
I20260808 22:26:44.817266 9372 ts_tablet_manager.cc:1939] T e77ab6b393bd45b4bdcef8cd29e83856 P 38ebf0799f9044a688c56196e95a4f8f: Deleting consensus metadata
I20260808 22:26:44.817524 9371 tablet_replica.cc:330] T e77ab6b393bd45b4bdcef8cd29e83856 P 362b664198bb4b25aacfd84c6f0d5119: stopping tablet replica
I20260808 22:26:44.817682 9371 raft_consensus.cc:2229] T e77ab6b393bd45b4bdcef8cd29e83856 P 362b664198bb4b25aacfd84c6f0d5119 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:26:44.817742 9373 ts_tablet_manager.cc:1918] T e77ab6b393bd45b4bdcef8cd29e83856 P b016084d0efc4659b75da627a2c14a6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 22:26:44.817816 9373 log.cc:1196] T e77ab6b393bd45b4bdcef8cd29e83856 P b016084d0efc4659b75da627a2c14a6e: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-2/wal/wals/e77ab6b393bd45b4bdcef8cd29e83856
I20260808 22:26:44.817806 9371 raft_consensus.cc:2258] T e77ab6b393bd45b4bdcef8cd29e83856 P 362b664198bb4b25aacfd84c6f0d5119 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:26:44.817920 8281 catalog_manager.cc:4841] TS 38ebf0799f9044a688c56196e95a4f8f (127.5.11.194:36057): tablet e77ab6b393bd45b4bdcef8cd29e83856 (table test-table [id=486d9cd6fb444986af7f3f00b8554180]) successfully deleted
I20260808 22:26:44.818126 9373 ts_tablet_manager.cc:1939] T e77ab6b393bd45b4bdcef8cd29e83856 P b016084d0efc4659b75da627a2c14a6e: Deleting consensus metadata
I20260808 22:26:44.818432 9371 ts_tablet_manager.cc:1905] T e77ab6b393bd45b4bdcef8cd29e83856 P 362b664198bb4b25aacfd84c6f0d5119: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:26:44.818755 8277 catalog_manager.cc:4841] TS b016084d0efc4659b75da627a2c14a6e (127.5.11.195:41455): tablet e77ab6b393bd45b4bdcef8cd29e83856 (table test-table [id=486d9cd6fb444986af7f3f00b8554180]) successfully deleted
I20260808 22:26:44.821070 9371 ts_tablet_manager.cc:1918] T e77ab6b393bd45b4bdcef8cd29e83856 P 362b664198bb4b25aacfd84c6f0d5119: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 22:26:44.821139 9371 log.cc:1196] T e77ab6b393bd45b4bdcef8cd29e83856 P 362b664198bb4b25aacfd84c6f0d5119: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-0/wal/wals/e77ab6b393bd45b4bdcef8cd29e83856
I20260808 22:26:44.821393 9371 ts_tablet_manager.cc:1939] T e77ab6b393bd45b4bdcef8cd29e83856 P 362b664198bb4b25aacfd84c6f0d5119: Deleting consensus metadata
I20260808 22:26:44.822139 8274 catalog_manager.cc:4841] TS 362b664198bb4b25aacfd84c6f0d5119 (127.5.11.193:44745): tablet e77ab6b393bd45b4bdcef8cd29e83856 (table test-table [id=486d9cd6fb444986af7f3f00b8554180]) successfully deleted
Aug 08 22:26:44 dist-test-slave-7f01 krb5kdc[8212](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228004, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 22:26:44 dist-test-slave-7f01 krb5kdc[8212](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228004, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:26:44.848750 9387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:44.836782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45140 (local address 127.5.11.254:44629)
0808 22:26:44.837043 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:44.837055 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:44.837176 (+ 121us) server_negotiation.cc:407] Connection header received
0808 22:26:44.837326 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:44.837328 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:44.837375 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:44.837467 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:44.838250 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:44.838897 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:44.839796 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:44.840100 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:44.840183 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:44.845780 (+ 5597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:44.845783 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:44.845785 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:44.845808 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:44.847151 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:44.847531 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:44.847533 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:44.847535 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:44.847615 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:44.847923 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:44.847926 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:44.847927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:44.848114 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:44.848262 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:44.848433 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 22:26:44.848553 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":96,"threads_started":1}
I20260808 22:26:44.851294 8305 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45140:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:26:44.851679 8305 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:26:44.856454 8559 tablet_service.cc:1467] Processing CreateTablet for tablet ee65b5b9f82d410789426e6007504c79 (DEFAULT_TABLE table=test-table [id=c22f0b5dd8944da59941bafe3d302aee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:26:44.856483 8803 tablet_service.cc:1467] Processing CreateTablet for tablet ee65b5b9f82d410789426e6007504c79 (DEFAULT_TABLE table=test-table [id=c22f0b5dd8944da59941bafe3d302aee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:26:44.856768 8803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee65b5b9f82d410789426e6007504c79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:44.856769 8559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee65b5b9f82d410789426e6007504c79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:44.856818 9069 tablet_service.cc:1467] Processing CreateTablet for tablet ee65b5b9f82d410789426e6007504c79 (DEFAULT_TABLE table=test-table [id=c22f0b5dd8944da59941bafe3d302aee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:26:44.857124 9069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ee65b5b9f82d410789426e6007504c79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:44.859392 9390 tablet_bootstrap.cc:492] T ee65b5b9f82d410789426e6007504c79 P b016084d0efc4659b75da627a2c14a6e: Bootstrap starting.
I20260808 22:26:44.859431 9391 tablet_bootstrap.cc:492] T ee65b5b9f82d410789426e6007504c79 P 362b664198bb4b25aacfd84c6f0d5119: Bootstrap starting.
I20260808 22:26:44.859932 9392 tablet_bootstrap.cc:492] T ee65b5b9f82d410789426e6007504c79 P 38ebf0799f9044a688c56196e95a4f8f: Bootstrap starting.
I20260808 22:26:44.860327 9391 tablet_bootstrap.cc:654] T ee65b5b9f82d410789426e6007504c79 P 362b664198bb4b25aacfd84c6f0d5119: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:44.860337 9390 tablet_bootstrap.cc:654] T ee65b5b9f82d410789426e6007504c79 P b016084d0efc4659b75da627a2c14a6e: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:44.860790 9392 tablet_bootstrap.cc:654] T ee65b5b9f82d410789426e6007504c79 P 38ebf0799f9044a688c56196e95a4f8f: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:44.861678 9390 tablet_bootstrap.cc:492] T ee65b5b9f82d410789426e6007504c79 P b016084d0efc4659b75da627a2c14a6e: No bootstrap required, opened a new log
I20260808 22:26:44.861790 9390 ts_tablet_manager.cc:1397] T ee65b5b9f82d410789426e6007504c79 P b016084d0efc4659b75da627a2c14a6e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 22:26:44.861934 9391 tablet_bootstrap.cc:492] T ee65b5b9f82d410789426e6007504c79 P 362b664198bb4b25aacfd84c6f0d5119: No bootstrap required, opened a new log
I20260808 22:26:44.862025 9391 ts_tablet_manager.cc:1397] T ee65b5b9f82d410789426e6007504c79 P 362b664198bb4b25aacfd84c6f0d5119: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:26:44.862085 9392 tablet_bootstrap.cc:492] T ee65b5b9f82d410789426e6007504c79 P 38ebf0799f9044a688c56196e95a4f8f: No bootstrap required, opened a new log
I20260808 22:26:44.862167 9392 ts_tablet_manager.cc:1397] T ee65b5b9f82d410789426e6007504c79 P 38ebf0799f9044a688c56196e95a4f8f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 22:26:44.862267 9390 raft_consensus.cc:348] T ee65b5b9f82d410789426e6007504c79 P b016084d0efc4659b75da627a2c14a6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } }
I20260808 22:26:44.862383 9390 raft_consensus.cc:374] T ee65b5b9f82d410789426e6007504c79 P b016084d0efc4659b75da627a2c14a6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:44.862423 9390 raft_consensus.cc:729] T ee65b5b9f82d410789426e6007504c79 P b016084d0efc4659b75da627a2c14a6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b016084d0efc4659b75da627a2c14a6e, State: Initialized, Role: FOLLOWER
I20260808 22:26:44.862479 9391 raft_consensus.cc:348] T ee65b5b9f82d410789426e6007504c79 P 362b664198bb4b25aacfd84c6f0d5119 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } }
I20260808 22:26:44.862534 9392 raft_consensus.cc:348] T ee65b5b9f82d410789426e6007504c79 P 38ebf0799f9044a688c56196e95a4f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } }
I20260808 22:26:44.862555 9390 consensus_queue.cc:260] T ee65b5b9f82d410789426e6007504c79 P b016084d0efc4659b75da627a2c14a6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } }
I20260808 22:26:44.862617 9391 raft_consensus.cc:374] T ee65b5b9f82d410789426e6007504c79 P 362b664198bb4b25aacfd84c6f0d5119 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:44.862629 9392 raft_consensus.cc:374] T ee65b5b9f82d410789426e6007504c79 P 38ebf0799f9044a688c56196e95a4f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:44.862658 9391 raft_consensus.cc:729] T ee65b5b9f82d410789426e6007504c79 P 362b664198bb4b25aacfd84c6f0d5119 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 362b664198bb4b25aacfd84c6f0d5119, State: Initialized, Role: FOLLOWER
I20260808 22:26:44.862665 9392 raft_consensus.cc:729] T ee65b5b9f82d410789426e6007504c79 P 38ebf0799f9044a688c56196e95a4f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38ebf0799f9044a688c56196e95a4f8f, State: Initialized, Role: FOLLOWER
I20260808 22:26:44.862793 9392 consensus_queue.cc:260] T ee65b5b9f82d410789426e6007504c79 P 38ebf0799f9044a688c56196e95a4f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } }
I20260808 22:26:44.862818 9391 consensus_queue.cc:260] T ee65b5b9f82d410789426e6007504c79 P 362b664198bb4b25aacfd84c6f0d5119 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } }
I20260808 22:26:44.862856 9390 ts_tablet_manager.cc:1428] T ee65b5b9f82d410789426e6007504c79 P b016084d0efc4659b75da627a2c14a6e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:26:44.863159 9392 ts_tablet_manager.cc:1428] T ee65b5b9f82d410789426e6007504c79 P 38ebf0799f9044a688c56196e95a4f8f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:26:44.863178 9391 ts_tablet_manager.cc:1428] T ee65b5b9f82d410789426e6007504c79 P 362b664198bb4b25aacfd84c6f0d5119: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 22:26:44.900363 9300 raft_consensus.cc:482] T ee65b5b9f82d410789426e6007504c79 P b016084d0efc4659b75da627a2c14a6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:26:44.900503 9300 raft_consensus.cc:504] T ee65b5b9f82d410789426e6007504c79 P b016084d0efc4659b75da627a2c14a6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } }
I20260808 22:26:44.900931 9300 leader_election.cc:288] T ee65b5b9f82d410789426e6007504c79 P b016084d0efc4659b75da627a2c14a6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 362b664198bb4b25aacfd84c6f0d5119 (127.5.11.193:44745), 38ebf0799f9044a688c56196e95a4f8f (127.5.11.194:36057)
I20260808 22:26:44.901333 8592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee65b5b9f82d410789426e6007504c79" candidate_uuid: "b016084d0efc4659b75da627a2c14a6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "362b664198bb4b25aacfd84c6f0d5119" is_pre_election: true
I20260808 22:26:44.901389 8850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee65b5b9f82d410789426e6007504c79" candidate_uuid: "b016084d0efc4659b75da627a2c14a6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38ebf0799f9044a688c56196e95a4f8f" is_pre_election: true
I20260808 22:26:44.901489 8592 raft_consensus.cc:2454] T ee65b5b9f82d410789426e6007504c79 P 362b664198bb4b25aacfd84c6f0d5119 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b016084d0efc4659b75da627a2c14a6e in term 0.
I20260808 22:26:44.901572 8850 raft_consensus.cc:2454] T ee65b5b9f82d410789426e6007504c79 P 38ebf0799f9044a688c56196e95a4f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b016084d0efc4659b75da627a2c14a6e in term 0.
I20260808 22:26:44.901835 8975 leader_election.cc:302] T ee65b5b9f82d410789426e6007504c79 P b016084d0efc4659b75da627a2c14a6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 362b664198bb4b25aacfd84c6f0d5119, b016084d0efc4659b75da627a2c14a6e; no voters:
I20260808 22:26:44.902020 9300 raft_consensus.cc:2781] T ee65b5b9f82d410789426e6007504c79 P b016084d0efc4659b75da627a2c14a6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:26:44.902081 9300 raft_consensus.cc:482] T ee65b5b9f82d410789426e6007504c79 P b016084d0efc4659b75da627a2c14a6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:26:44.902109 9300 raft_consensus.cc:3037] T ee65b5b9f82d410789426e6007504c79 P b016084d0efc4659b75da627a2c14a6e [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:44.903105 9300 raft_consensus.cc:504] T ee65b5b9f82d410789426e6007504c79 P b016084d0efc4659b75da627a2c14a6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } }
I20260808 22:26:44.903455 9300 leader_election.cc:288] T ee65b5b9f82d410789426e6007504c79 P b016084d0efc4659b75da627a2c14a6e [CANDIDATE]: Term 1 election: Requested vote from peers 362b664198bb4b25aacfd84c6f0d5119 (127.5.11.193:44745), 38ebf0799f9044a688c56196e95a4f8f (127.5.11.194:36057)
I20260808 22:26:44.903834 8592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee65b5b9f82d410789426e6007504c79" candidate_uuid: "b016084d0efc4659b75da627a2c14a6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "362b664198bb4b25aacfd84c6f0d5119"
I20260808 22:26:44.903857 8850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ee65b5b9f82d410789426e6007504c79" candidate_uuid: "b016084d0efc4659b75da627a2c14a6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38ebf0799f9044a688c56196e95a4f8f"
I20260808 22:26:44.903955 8592 raft_consensus.cc:3037] T ee65b5b9f82d410789426e6007504c79 P 362b664198bb4b25aacfd84c6f0d5119 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:44.903955 8850 raft_consensus.cc:3037] T ee65b5b9f82d410789426e6007504c79 P 38ebf0799f9044a688c56196e95a4f8f [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:44.905053 8850 raft_consensus.cc:2454] T ee65b5b9f82d410789426e6007504c79 P 38ebf0799f9044a688c56196e95a4f8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b016084d0efc4659b75da627a2c14a6e in term 1.
I20260808 22:26:44.905053 8592 raft_consensus.cc:2454] T ee65b5b9f82d410789426e6007504c79 P 362b664198bb4b25aacfd84c6f0d5119 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b016084d0efc4659b75da627a2c14a6e in term 1.
I20260808 22:26:44.905368 8975 leader_election.cc:302] T ee65b5b9f82d410789426e6007504c79 P b016084d0efc4659b75da627a2c14a6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 362b664198bb4b25aacfd84c6f0d5119, b016084d0efc4659b75da627a2c14a6e; no voters:
I20260808 22:26:44.905545 9300 raft_consensus.cc:2781] T ee65b5b9f82d410789426e6007504c79 P b016084d0efc4659b75da627a2c14a6e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:26:44.905637 9300 raft_consensus.cc:686] T ee65b5b9f82d410789426e6007504c79 P b016084d0efc4659b75da627a2c14a6e [term 1 LEADER]: Becoming Leader. State: Replica: b016084d0efc4659b75da627a2c14a6e, State: Running, Role: LEADER
I20260808 22:26:44.905776 9300 consensus_queue.cc:237] T ee65b5b9f82d410789426e6007504c79 P b016084d0efc4659b75da627a2c14a6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } }
I20260808 22:26:44.907374 8303 catalog_manager.cc:5495] T ee65b5b9f82d410789426e6007504c79 P b016084d0efc4659b75da627a2c14a6e reported cstate change: term changed from 0 to 1, leader changed from <none> to b016084d0efc4659b75da627a2c14a6e (127.5.11.195). New cstate: current_term: 1 leader_uuid: "b016084d0efc4659b75da627a2c14a6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } health_report { overall_health: HEALTHY } } }
I20260808 22:26:44.931187 9407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:44.928188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41976 (local address 127.5.11.195:41455)
0808 22:26:44.928501 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:44.928508 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:44.928521 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:26:44.928560 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:44.928562 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:44.928602 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:44.928674 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:26:44.928926 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:44.929419 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:44.930218 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:44.930425 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:44.930507 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:44.930721 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:26:44.930791 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:26:44.930842 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:44.930934 (+ 92us) server_negotiation.cc:299] Negotiation successful
0808 22:26:44.930977 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":157,"threads_started":1}
I20260808 22:26:44.939953 9410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:44.936824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:41455 (local address 127.5.11.195:59295)
0808 22:26:44.937106 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:44.937119 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:44.937202 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:44.937433 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:44.937439 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:44.937449 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:26:44.937582 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:44.937587 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:44.938239 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:44.938242 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:44.938870 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:44.938879 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:44.939359 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:44.939362 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:44.939420 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:44.939444 (+ 24us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:26:44.939710 (+ 266us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:26:44.939716 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 22:26:44.939786 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 22:26:44.939845 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":20,"thread_start_us":117,"threads_started":1}
I20260808 22:26:44.940060 9407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:44.937002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:59295 (local address 127.5.11.195:41455)
0808 22:26:44.937113 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:44.937116 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:44.937181 (+ 65us) server_negotiation.cc:407] Connection header received
0808 22:26:44.937262 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:44.937264 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:44.937302 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:44.937391 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:26:44.937668 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:44.938145 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:44.938982 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:44.939251 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:44.939339 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:44.939543 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:26:44.939600 (+ 57us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:26:44.939669 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:44.939852 (+ 183us) server_negotiation.cc:299] Negotiation successful
0808 22:26:44.939896 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 22:26:44.945397 8850 raft_consensus.cc:1261] T ee65b5b9f82d410789426e6007504c79 P 38ebf0799f9044a688c56196e95a4f8f [term 1 FOLLOWER]: Refusing update from remote peer b016084d0efc4659b75da627a2c14a6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:26:44.945397 8592 raft_consensus.cc:1261] T ee65b5b9f82d410789426e6007504c79 P 362b664198bb4b25aacfd84c6f0d5119 [term 1 FOLLOWER]: Refusing update from remote peer b016084d0efc4659b75da627a2c14a6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:26:44.945770 9338 consensus_queue.cc:1035] T ee65b5b9f82d410789426e6007504c79 P b016084d0efc4659b75da627a2c14a6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:26:44.945928 9300 consensus_queue.cc:1035] T ee65b5b9f82d410789426e6007504c79 P b016084d0efc4659b75da627a2c14a6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:26:44.979334 8304 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:26:44.979542 8304 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:26:44.981597 8304 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 08a9e7267a0246319859b8084419fecb: Sending DeleteTablet for 3 replicas of tablet ee65b5b9f82d410789426e6007504c79
I20260808 22:26:44.982178 8803 tablet_service.cc:1514] Processing DeleteTablet for tablet ee65b5b9f82d410789426e6007504c79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:26:44 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:50598
I20260808 22:26:44.982178 8559 tablet_service.cc:1514] Processing DeleteTablet for tablet ee65b5b9f82d410789426e6007504c79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:26:44 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:58640
I20260808 22:26:44.982286 9067 tablet_service.cc:1514] Processing DeleteTablet for tablet ee65b5b9f82d410789426e6007504c79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:26:44 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:41954
I20260808 22:26:44.982354 9371 tablet_replica.cc:330] T ee65b5b9f82d410789426e6007504c79 P 362b664198bb4b25aacfd84c6f0d5119: stopping tablet replica
I20260808 22:26:44.982354 9372 tablet_replica.cc:330] T ee65b5b9f82d410789426e6007504c79 P 38ebf0799f9044a688c56196e95a4f8f: stopping tablet replica
I20260808 22:26:44.982460 9373 tablet_replica.cc:330] T ee65b5b9f82d410789426e6007504c79 P b016084d0efc4659b75da627a2c14a6e: stopping tablet replica
I20260808 22:26:44.982455 9372 raft_consensus.cc:2229] T ee65b5b9f82d410789426e6007504c79 P 38ebf0799f9044a688c56196e95a4f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:26:44.982455 9371 raft_consensus.cc:2229] T ee65b5b9f82d410789426e6007504c79 P 362b664198bb4b25aacfd84c6f0d5119 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:26:44.982553 9373 raft_consensus.cc:2229] T ee65b5b9f82d410789426e6007504c79 P b016084d0efc4659b75da627a2c14a6e [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:26:44.982563 9371 raft_consensus.cc:2258] T ee65b5b9f82d410789426e6007504c79 P 362b664198bb4b25aacfd84c6f0d5119 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:26:44.982563 9372 raft_consensus.cc:2258] T ee65b5b9f82d410789426e6007504c79 P 38ebf0799f9044a688c56196e95a4f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:26:44.982857 9373 raft_consensus.cc:2258] T ee65b5b9f82d410789426e6007504c79 P b016084d0efc4659b75da627a2c14a6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:26:44.983354 9371 ts_tablet_manager.cc:1905] T ee65b5b9f82d410789426e6007504c79 P 362b664198bb4b25aacfd84c6f0d5119: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:26:44.983353 9372 ts_tablet_manager.cc:1905] T ee65b5b9f82d410789426e6007504c79 P 38ebf0799f9044a688c56196e95a4f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:26:44.983583 9373 ts_tablet_manager.cc:1905] T ee65b5b9f82d410789426e6007504c79 P b016084d0efc4659b75da627a2c14a6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:26:44.985872 9372 ts_tablet_manager.cc:1918] T ee65b5b9f82d410789426e6007504c79 P 38ebf0799f9044a688c56196e95a4f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:26:44.985869 9371 ts_tablet_manager.cc:1918] T ee65b5b9f82d410789426e6007504c79 P 362b664198bb4b25aacfd84c6f0d5119: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:26:44.985944 9371 log.cc:1196] T ee65b5b9f82d410789426e6007504c79 P 362b664198bb4b25aacfd84c6f0d5119: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-0/wal/wals/ee65b5b9f82d410789426e6007504c79
I20260808 22:26:44.985944 9372 log.cc:1196] T ee65b5b9f82d410789426e6007504c79 P 38ebf0799f9044a688c56196e95a4f8f: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-1/wal/wals/ee65b5b9f82d410789426e6007504c79
I20260808 22:26:44.986132 9373 ts_tablet_manager.cc:1918] T ee65b5b9f82d410789426e6007504c79 P b016084d0efc4659b75da627a2c14a6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:26:44.986203 9373 log.cc:1196] T ee65b5b9f82d410789426e6007504c79 P b016084d0efc4659b75da627a2c14a6e: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-2/wal/wals/ee65b5b9f82d410789426e6007504c79
I20260808 22:26:44.986214 9371 ts_tablet_manager.cc:1939] T ee65b5b9f82d410789426e6007504c79 P 362b664198bb4b25aacfd84c6f0d5119: Deleting consensus metadata
I20260808 22:26:44.986214 9372 ts_tablet_manager.cc:1939] T ee65b5b9f82d410789426e6007504c79 P 38ebf0799f9044a688c56196e95a4f8f: Deleting consensus metadata
I20260808 22:26:44.986464 9373 ts_tablet_manager.cc:1939] T ee65b5b9f82d410789426e6007504c79 P b016084d0efc4659b75da627a2c14a6e: Deleting consensus metadata
I20260808 22:26:44.986852 8281 catalog_manager.cc:4841] TS 38ebf0799f9044a688c56196e95a4f8f (127.5.11.194:36057): tablet ee65b5b9f82d410789426e6007504c79 (table test-table [id=c22f0b5dd8944da59941bafe3d302aee]) successfully deleted
I20260808 22:26:44.986897 8274 catalog_manager.cc:4841] TS 362b664198bb4b25aacfd84c6f0d5119 (127.5.11.193:44745): tablet ee65b5b9f82d410789426e6007504c79 (table test-table [id=c22f0b5dd8944da59941bafe3d302aee]) successfully deleted
I20260808 22:26:44.987076 8277 catalog_manager.cc:4841] TS b016084d0efc4659b75da627a2c14a6e (127.5.11.195:41455): tablet ee65b5b9f82d410789426e6007504c79 (table test-table [id=c22f0b5dd8944da59941bafe3d302aee]) successfully deleted
I20260808 22:26:44.988039 8304 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45140:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:26:44.988452 8304 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:26:44.993075 8559 tablet_service.cc:1467] Processing CreateTablet for tablet 4c9e583b28dc4d4caf7f94a5a0cd3d6f (DEFAULT_TABLE table=test-table [id=96f61da1911b417cb718107361be2441]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:26:44.993243 9067 tablet_service.cc:1467] Processing CreateTablet for tablet 4c9e583b28dc4d4caf7f94a5a0cd3d6f (DEFAULT_TABLE table=test-table [id=96f61da1911b417cb718107361be2441]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:26:44.993363 8559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c9e583b28dc4d4caf7f94a5a0cd3d6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:44.993489 9067 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c9e583b28dc4d4caf7f94a5a0cd3d6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:44.995294 9391 tablet_bootstrap.cc:492] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P 362b664198bb4b25aacfd84c6f0d5119: Bootstrap starting.
I20260808 22:26:44.995668 9390 tablet_bootstrap.cc:492] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P b016084d0efc4659b75da627a2c14a6e: Bootstrap starting.
I20260808 22:26:44.995951 8803 tablet_service.cc:1467] Processing CreateTablet for tablet 4c9e583b28dc4d4caf7f94a5a0cd3d6f (DEFAULT_TABLE table=test-table [id=96f61da1911b417cb718107361be2441]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:26:44.996213 8803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4c9e583b28dc4d4caf7f94a5a0cd3d6f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:44.996356 9391 tablet_bootstrap.cc:654] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P 362b664198bb4b25aacfd84c6f0d5119: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:44.996531 9390 tablet_bootstrap.cc:654] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P b016084d0efc4659b75da627a2c14a6e: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:44.998288 9390 tablet_bootstrap.cc:492] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P b016084d0efc4659b75da627a2c14a6e: No bootstrap required, opened a new log
I20260808 22:26:44.998363 9390 ts_tablet_manager.cc:1397] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P b016084d0efc4659b75da627a2c14a6e: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260808 22:26:44.998482 9392 tablet_bootstrap.cc:492] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P 38ebf0799f9044a688c56196e95a4f8f: Bootstrap starting.
I20260808 22:26:44.998746 9390 raft_consensus.cc:348] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P b016084d0efc4659b75da627a2c14a6e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } }
I20260808 22:26:44.998875 9390 raft_consensus.cc:374] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P b016084d0efc4659b75da627a2c14a6e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:44.998917 9390 raft_consensus.cc:729] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P b016084d0efc4659b75da627a2c14a6e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b016084d0efc4659b75da627a2c14a6e, State: Initialized, Role: FOLLOWER
I20260808 22:26:44.999037 9390 consensus_queue.cc:260] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P b016084d0efc4659b75da627a2c14a6e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } }
I20260808 22:26:44.999284 9390 ts_tablet_manager.cc:1428] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P b016084d0efc4659b75da627a2c14a6e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:26:44.999456 9392 tablet_bootstrap.cc:654] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P 38ebf0799f9044a688c56196e95a4f8f: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:45.000375 9391 tablet_bootstrap.cc:492] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P 362b664198bb4b25aacfd84c6f0d5119: No bootstrap required, opened a new log
I20260808 22:26:45.000478 9391 ts_tablet_manager.cc:1397] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P 362b664198bb4b25aacfd84c6f0d5119: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260808 22:26:45.000941 9391 raft_consensus.cc:348] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P 362b664198bb4b25aacfd84c6f0d5119 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } }
I20260808 22:26:45.001070 9391 raft_consensus.cc:374] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P 362b664198bb4b25aacfd84c6f0d5119 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:45.001111 9391 raft_consensus.cc:729] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P 362b664198bb4b25aacfd84c6f0d5119 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 362b664198bb4b25aacfd84c6f0d5119, State: Initialized, Role: FOLLOWER
I20260808 22:26:45.001247 9391 consensus_queue.cc:260] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P 362b664198bb4b25aacfd84c6f0d5119 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } }
I20260808 22:26:45.001540 9391 ts_tablet_manager.cc:1428] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P 362b664198bb4b25aacfd84c6f0d5119: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260808 22:26:45.002529 9392 tablet_bootstrap.cc:492] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P 38ebf0799f9044a688c56196e95a4f8f: No bootstrap required, opened a new log
I20260808 22:26:45.002595 9392 ts_tablet_manager.cc:1397] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P 38ebf0799f9044a688c56196e95a4f8f: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260808 22:26:45.002871 9392 raft_consensus.cc:348] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P 38ebf0799f9044a688c56196e95a4f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } }
I20260808 22:26:45.002943 9392 raft_consensus.cc:374] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P 38ebf0799f9044a688c56196e95a4f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:45.003000 9392 raft_consensus.cc:729] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P 38ebf0799f9044a688c56196e95a4f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38ebf0799f9044a688c56196e95a4f8f, State: Initialized, Role: FOLLOWER
I20260808 22:26:45.003122 9392 consensus_queue.cc:260] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P 38ebf0799f9044a688c56196e95a4f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } }
I20260808 22:26:45.003399 9392 ts_tablet_manager.cc:1428] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P 38ebf0799f9044a688c56196e95a4f8f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 22:26:45.385268 9345 raft_consensus.cc:482] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P b016084d0efc4659b75da627a2c14a6e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:26:45.385404 9345 raft_consensus.cc:504] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P b016084d0efc4659b75da627a2c14a6e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } }
I20260808 22:26:45.385859 9345 leader_election.cc:288] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P b016084d0efc4659b75da627a2c14a6e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 362b664198bb4b25aacfd84c6f0d5119 (127.5.11.193:44745), 38ebf0799f9044a688c56196e95a4f8f (127.5.11.194:36057)
I20260808 22:26:45.386354 8850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c9e583b28dc4d4caf7f94a5a0cd3d6f" candidate_uuid: "b016084d0efc4659b75da627a2c14a6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38ebf0799f9044a688c56196e95a4f8f" is_pre_election: true
I20260808 22:26:45.386436 8592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c9e583b28dc4d4caf7f94a5a0cd3d6f" candidate_uuid: "b016084d0efc4659b75da627a2c14a6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "362b664198bb4b25aacfd84c6f0d5119" is_pre_election: true
I20260808 22:26:45.386535 8850 raft_consensus.cc:2454] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P 38ebf0799f9044a688c56196e95a4f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b016084d0efc4659b75da627a2c14a6e in term 0.
I20260808 22:26:45.386643 8592 raft_consensus.cc:2454] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P 362b664198bb4b25aacfd84c6f0d5119 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b016084d0efc4659b75da627a2c14a6e in term 0.
I20260808 22:26:45.386931 8978 leader_election.cc:302] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P b016084d0efc4659b75da627a2c14a6e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 38ebf0799f9044a688c56196e95a4f8f, b016084d0efc4659b75da627a2c14a6e; no voters:
I20260808 22:26:45.387156 9345 raft_consensus.cc:2781] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P b016084d0efc4659b75da627a2c14a6e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:26:45.387234 9345 raft_consensus.cc:482] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P b016084d0efc4659b75da627a2c14a6e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:26:45.387290 9345 raft_consensus.cc:3037] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P b016084d0efc4659b75da627a2c14a6e [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:45.388443 9345 raft_consensus.cc:504] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P b016084d0efc4659b75da627a2c14a6e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } }
I20260808 22:26:45.388763 9345 leader_election.cc:288] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P b016084d0efc4659b75da627a2c14a6e [CANDIDATE]: Term 1 election: Requested vote from peers 362b664198bb4b25aacfd84c6f0d5119 (127.5.11.193:44745), 38ebf0799f9044a688c56196e95a4f8f (127.5.11.194:36057)
I20260808 22:26:45.389150 8592 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c9e583b28dc4d4caf7f94a5a0cd3d6f" candidate_uuid: "b016084d0efc4659b75da627a2c14a6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "362b664198bb4b25aacfd84c6f0d5119"
I20260808 22:26:45.389153 8850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4c9e583b28dc4d4caf7f94a5a0cd3d6f" candidate_uuid: "b016084d0efc4659b75da627a2c14a6e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "38ebf0799f9044a688c56196e95a4f8f"
I20260808 22:26:45.389247 8592 raft_consensus.cc:3037] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P 362b664198bb4b25aacfd84c6f0d5119 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:45.389297 8850 raft_consensus.cc:3037] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P 38ebf0799f9044a688c56196e95a4f8f [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:45.390156 8592 raft_consensus.cc:2454] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P 362b664198bb4b25aacfd84c6f0d5119 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b016084d0efc4659b75da627a2c14a6e in term 1.
I20260808 22:26:45.390301 8850 raft_consensus.cc:2454] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P 38ebf0799f9044a688c56196e95a4f8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b016084d0efc4659b75da627a2c14a6e in term 1.
I20260808 22:26:45.390448 8975 leader_election.cc:302] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P b016084d0efc4659b75da627a2c14a6e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 362b664198bb4b25aacfd84c6f0d5119, b016084d0efc4659b75da627a2c14a6e; no voters:
I20260808 22:26:45.390617 9345 raft_consensus.cc:2781] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P b016084d0efc4659b75da627a2c14a6e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:26:45.390702 9345 raft_consensus.cc:686] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P b016084d0efc4659b75da627a2c14a6e [term 1 LEADER]: Becoming Leader. State: Replica: b016084d0efc4659b75da627a2c14a6e, State: Running, Role: LEADER
I20260808 22:26:45.390889 9345 consensus_queue.cc:237] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P b016084d0efc4659b75da627a2c14a6e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } }
I20260808 22:26:45.392516 8304 catalog_manager.cc:5495] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P b016084d0efc4659b75da627a2c14a6e reported cstate change: term changed from 0 to 1, leader changed from <none> to b016084d0efc4659b75da627a2c14a6e (127.5.11.195). New cstate: current_term: 1 leader_uuid: "b016084d0efc4659b75da627a2c14a6e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b016084d0efc4659b75da627a2c14a6e" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 41455 } health_report { overall_health: HEALTHY } } }
I20260808 22:26:45.461529 8850 raft_consensus.cc:1261] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P 38ebf0799f9044a688c56196e95a4f8f [term 1 FOLLOWER]: Refusing update from remote peer b016084d0efc4659b75da627a2c14a6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:26:45.461849 8592 raft_consensus.cc:1261] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P 362b664198bb4b25aacfd84c6f0d5119 [term 1 FOLLOWER]: Refusing update from remote peer b016084d0efc4659b75da627a2c14a6e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:26:45.461913 9300 consensus_queue.cc:1035] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P b016084d0efc4659b75da627a2c14a6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "38ebf0799f9044a688c56196e95a4f8f" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:26:45.462280 9300 consensus_queue.cc:1035] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P b016084d0efc4659b75da627a2c14a6e [LEADER]: Connected to new peer: Peer: permanent_uuid: "362b664198bb4b25aacfd84c6f0d5119" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 44745 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 22:26:45.475713 9335 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260808 22:26:47.025416 8304 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:26:47.025570 8304 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45140:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:26:47.027925 8304 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 08a9e7267a0246319859b8084419fecb: Sending DeleteTablet for 3 replicas of tablet 4c9e583b28dc4d4caf7f94a5a0cd3d6f
I20260808 22:26:47.028678 8803 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c9e583b28dc4d4caf7f94a5a0cd3d6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:26:47 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:50598
I20260808 22:26:47.028785 9069 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c9e583b28dc4d4caf7f94a5a0cd3d6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:26:47 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:41954
I20260808 22:26:47.028800 8559 tablet_service.cc:1514] Processing DeleteTablet for tablet 4c9e583b28dc4d4caf7f94a5a0cd3d6f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:26:47 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:58640
I20260808 22:26:47.028981 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 8449
I20260808 22:26:47.036964 9460 tablet_replica.cc:330] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P 38ebf0799f9044a688c56196e95a4f8f: stopping tablet replica
I20260808 22:26:47.037137 9460 raft_consensus.cc:2229] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P 38ebf0799f9044a688c56196e95a4f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:26:47.037165 9462 tablet_replica.cc:330] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P b016084d0efc4659b75da627a2c14a6e: stopping tablet replica
I20260808 22:26:47.037267 9460 raft_consensus.cc:2258] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P 38ebf0799f9044a688c56196e95a4f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:26:47.037274 9462 raft_consensus.cc:2229] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P b016084d0efc4659b75da627a2c14a6e [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:26:47.037479 9462 raft_consensus.cc:2258] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P b016084d0efc4659b75da627a2c14a6e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:26:47.037968 9460 ts_tablet_manager.cc:1905] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P 38ebf0799f9044a688c56196e95a4f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:26:47.038435 9462 ts_tablet_manager.cc:1905] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P b016084d0efc4659b75da627a2c14a6e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:26:47.040997 9460 ts_tablet_manager.cc:1918] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P 38ebf0799f9044a688c56196e95a4f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 22:26:47.041093 9460 log.cc:1196] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P 38ebf0799f9044a688c56196e95a4f8f: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-1/wal/wals/4c9e583b28dc4d4caf7f94a5a0cd3d6f
I20260808 22:26:47.041396 9460 ts_tablet_manager.cc:1939] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P 38ebf0799f9044a688c56196e95a4f8f: Deleting consensus metadata
I20260808 22:26:47.042052 9462 ts_tablet_manager.cc:1918] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P b016084d0efc4659b75da627a2c14a6e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260808 22:26:47.042102 8281 catalog_manager.cc:4841] TS 38ebf0799f9044a688c56196e95a4f8f (127.5.11.194:36057): tablet 4c9e583b28dc4d4caf7f94a5a0cd3d6f (table test-table [id=96f61da1911b417cb718107361be2441]) successfully deleted
I20260808 22:26:47.042158 9462 log.cc:1196] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P b016084d0efc4659b75da627a2c14a6e: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786227991988852-5167-0/minicluster-data/ts-2/wal/wals/4c9e583b28dc4d4caf7f94a5a0cd3d6f
I20260808 22:26:47.042456 9462 ts_tablet_manager.cc:1939] T 4c9e583b28dc4d4caf7f94a5a0cd3d6f P b016084d0efc4659b75da627a2c14a6e: Deleting consensus metadata
I20260808 22:26:47.043042 8277 catalog_manager.cc:4841] TS b016084d0efc4659b75da627a2c14a6e (127.5.11.195:41455): tablet 4c9e583b28dc4d4caf7f94a5a0cd3d6f (table test-table [id=96f61da1911b417cb718107361be2441]) successfully deleted
W20260808 22:26:47.045782 8274 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.11.193:44745 (error 108)
I20260808 22:26:47.046137 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 8694
I20260808 22:26:47.048357 9468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:47.047885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:44745 (local address 127.0.0.1:58646)
0808 22:26:47.048141 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:47.048261 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.193:44745: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":90,"threads_started":1}
W20260808 22:26:47.048604 8274 catalog_manager.cc:4584] TS 362b664198bb4b25aacfd84c6f0d5119 (127.5.11.193:44745): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4c9e583b28dc4d4caf7f94a5a0cd3d6f: Network error: Client connection negotiation failed: client connection to 127.5.11.193:44745: connect: Connection refused (error 111)
I20260808 22:26:47.059846 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 8943
I20260808 22:26:47.073868 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 8233
2026-08-08T22:26:47Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6739 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:26:47 dist-test-slave-7f01 krb5kdc[9477](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:26:47 dist-test-slave-7f01 krb5kdc[9477](info): set up 2 sockets
Aug 08 22:26:47 dist-test-slave-7f01 krb5kdc[9477](info): commencing operation
krb5kdc: starting...
W20260808 22:26:48.321645 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.220s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:26:48 dist-test-slave-7f01 krb5kdc[9477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:26:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:26:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.11.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
I20260808 22:26:48.451725 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:46445
--webserver_interface=127.5.11.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:45463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:46445
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:26:48.562791 9506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:26:48.563118 9506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:26:48.563199 9506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:26:48.567330 9506 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:26:48.567427 9506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:26:48.567467 9506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:26:48.567524 9506 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:26:48.567564 9506 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:26:48.572894 9506 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:45463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:46445
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:46445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:26:48.573932 9506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:26:48.574967 9506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:26:48.583835 9517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:48.583940 9506 server_base.cc:1034] running on GCE node
W20260808 22:26:48.583791 9523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:48.583835 9516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:48.584738 9506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:26:48.585800 9506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:26:48.587013 9506 hybrid_clock.cc:648] HybridClock initialized: now 1786228008586990 us; error 35 us; skew 500 ppm
Aug 08 22:26:48 dist-test-slave-7f01 krb5kdc[9477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228008, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:26:48.589979 9506 init.cc:377] Logged in from keytab as kudu/127.5.11.254@KRBTEST.COM (short username kudu)
I20260808 22:26:48.590893 9506 webserver.cc:460] Webserver started at http://127.5.11.254:46313/ using document root <none> and password file <none>
I20260808 22:26:48.591490 9506 fs_manager.cc:359] Metadata directory not provided
I20260808 22:26:48.591593 9506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:26:48.591894 9506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:26:48.594045 9506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "87ffb15514bd48b186d0976ae2016a25"
format_stamp: "Formatted at 2026-08-08 22:26:48 on dist-test-slave-7f01"
server_key: "78eabc3fded2b2e1a49e27f9990d0572"
server_key_iv: "0d60a33c8c12a92deb49c3e414c9c8e1"
server_key_version: "encryptionkey@0"
I20260808 22:26:48.594650 9506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "87ffb15514bd48b186d0976ae2016a25"
format_stamp: "Formatted at 2026-08-08 22:26:48 on dist-test-slave-7f01"
server_key: "78eabc3fded2b2e1a49e27f9990d0572"
server_key_iv: "0d60a33c8c12a92deb49c3e414c9c8e1"
server_key_version: "encryptionkey@0"
I20260808 22:26:48.598619 9506 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 22:26:48.601125 9532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:48.602406 9506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:26:48.602555 9506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "87ffb15514bd48b186d0976ae2016a25"
format_stamp: "Formatted at 2026-08-08 22:26:48 on dist-test-slave-7f01"
server_key: "78eabc3fded2b2e1a49e27f9990d0572"
server_key_iv: "0d60a33c8c12a92deb49c3e414c9c8e1"
server_key_version: "encryptionkey@0"
I20260808 22:26:48.602670 9506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:26:48.621338 9506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:26:48.624760 9506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:26:48.624986 9506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:26:48.632253 9506 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:46445
I20260808 22:26:48.632269 9627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:46445 every 8 connection(s)
I20260808 22:26:48.633248 9506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:26:48.636178 9628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:48.637535 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 9506
I20260808 22:26:48.637632 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:26:48.637923 5167 external_mini_cluster.cc:1428] Setting key 52c09615f4f898cb8eb40dd3b3272f58
I20260808 22:26:48.642261 9628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87ffb15514bd48b186d0976ae2016a25: Bootstrap starting.
I20260808 22:26:48.644573 9628 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87ffb15514bd48b186d0976ae2016a25: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:48.645313 9628 log.cc:824] T 00000000000000000000000000000000 P 87ffb15514bd48b186d0976ae2016a25: Log is configured to *not* fsync() on all Append() calls
Aug 08 22:26:48 dist-test-slave-7f01 krb5kdc[9477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228008, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:26:48.647296 9628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87ffb15514bd48b186d0976ae2016a25: No bootstrap required, opened a new log
I20260808 22:26:48.650810 9628 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87ffb15514bd48b186d0976ae2016a25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87ffb15514bd48b186d0976ae2016a25" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 46445 } }
I20260808 22:26:48.651059 9628 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87ffb15514bd48b186d0976ae2016a25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:48.651116 9628 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87ffb15514bd48b186d0976ae2016a25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87ffb15514bd48b186d0976ae2016a25, State: Initialized, Role: FOLLOWER
I20260808 22:26:48.651580 9628 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87ffb15514bd48b186d0976ae2016a25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87ffb15514bd48b186d0976ae2016a25" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 46445 } }
I20260808 22:26:48.651742 9628 raft_consensus.cc:388] T 00000000000000000000000000000000 P 87ffb15514bd48b186d0976ae2016a25 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:26:48.651857 9628 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87ffb15514bd48b186d0976ae2016a25 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:26:48.651984 9628 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87ffb15514bd48b186d0976ae2016a25 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:48.652287 9638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:48.639410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35540 (local address 127.5.11.254:46445)
0808 22:26:48.639893 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:48.639909 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:48.639945 (+ 36us) server_negotiation.cc:407] Connection header received
0808 22:26:48.640593 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:48.640610 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:48.640937 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:48.641279 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:48.642113 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:48.643166 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:48.644081 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:48.644570 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:48.644746 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:48.647293 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:48.647298 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:48.647312 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:48.647339 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:48.649173 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:48.649696 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:48.649701 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:48.649705 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:48.649832 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:48.650132 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:48.650136 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:48.650138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:48.650543 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:48.650773 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:48.651053 (+ 280us) server_negotiation.cc:299] Negotiation successful
0808 22:26:48.651379 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":122,"threads_started":1}
I20260808 22:26:48.652953 9628 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87ffb15514bd48b186d0976ae2016a25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87ffb15514bd48b186d0976ae2016a25" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 46445 } }
I20260808 22:26:48.653337 9628 leader_election.cc:302] T 00000000000000000000000000000000 P 87ffb15514bd48b186d0976ae2016a25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 87ffb15514bd48b186d0976ae2016a25; no voters:
I20260808 22:26:48.653674 9628 leader_election.cc:288] T 00000000000000000000000000000000 P 87ffb15514bd48b186d0976ae2016a25 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:26:48.653795 9640 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87ffb15514bd48b186d0976ae2016a25 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:26:48.654026 9640 raft_consensus.cc:686] T 00000000000000000000000000000000 P 87ffb15514bd48b186d0976ae2016a25 [term 1 LEADER]: Becoming Leader. State: Replica: 87ffb15514bd48b186d0976ae2016a25, State: Running, Role: LEADER
I20260808 22:26:48.654332 9640 consensus_queue.cc:237] T 00000000000000000000000000000000 P 87ffb15514bd48b186d0976ae2016a25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87ffb15514bd48b186d0976ae2016a25" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 46445 } }
I20260808 22:26:48.654968 9628 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87ffb15514bd48b186d0976ae2016a25 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:26:48.655558 9640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87ffb15514bd48b186d0976ae2016a25 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 87ffb15514bd48b186d0976ae2016a25. Latest consensus state: current_term: 1 leader_uuid: "87ffb15514bd48b186d0976ae2016a25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87ffb15514bd48b186d0976ae2016a25" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 46445 } } }
I20260808 22:26:48.655692 9640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87ffb15514bd48b186d0976ae2016a25 [sys.catalog]: This master's current role is: LEADER
I20260808 22:26:48.656198 9641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87ffb15514bd48b186d0976ae2016a25 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "87ffb15514bd48b186d0976ae2016a25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "87ffb15514bd48b186d0976ae2016a25" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 46445 } } }
I20260808 22:26:48.656308 9641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87ffb15514bd48b186d0976ae2016a25 [sys.catalog]: This master's current role is: LEADER
I20260808 22:26:48.657490 9647 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:26:48.660676 9647 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:26:48.665961 9647 catalog_manager.cc:1347] Generated new cluster ID: 86b6629cd9054a709006f394d3d31ece
I20260808 22:26:48.666033 9647 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:26:48.685125 9647 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:26:48.686484 9647 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:26:48.697302 9647 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 87ffb15514bd48b186d0976ae2016a25: Generated new TSK 0
I20260808 22:26:48.697978 9647 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.11.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab.
Entry for principal kudu/127.5.11.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab.
I20260808 22:26:48.735127 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.193:0
--local_ip_for_outbound_sockets=127.5.11.193
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:46445
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:45463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:26:48.848996 9672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:26:48.849242 9672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:26:48.849315 9672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:26:48.853848 9672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:26:48.853924 9672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:26:48.854012 9672 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.193
I20260808 22:26:48.859543 9672 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:45463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:46445
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.193
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:26:48.860664 9672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:26:48.861687 9672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:26:48.871124 9682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:48.871170 9686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:48.871124 9683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:48.871351 9672 server_base.cc:1034] running on GCE node
I20260808 22:26:48.871872 9672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:26:48.872473 9672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:26:48.873631 9672 hybrid_clock.cc:648] HybridClock initialized: now 1786228008873612 us; error 29 us; skew 500 ppm
Aug 08 22:26:48 dist-test-slave-7f01 krb5kdc[9477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228008, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:26:48.876623 9672 init.cc:377] Logged in from keytab as kudu/127.5.11.193@KRBTEST.COM (short username kudu)
I20260808 22:26:48.877785 9672 webserver.cc:460] Webserver started at http://127.5.11.193:34511/ using document root <none> and password file <none>
I20260808 22:26:48.878535 9672 fs_manager.cc:359] Metadata directory not provided
I20260808 22:26:48.878643 9672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:26:48.878868 9672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:26:48.881984 9672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-0/data/instance:
uuid: "ae8e7afb3c91475db96e9811fe0a32d5"
format_stamp: "Formatted at 2026-08-08 22:26:48 on dist-test-slave-7f01"
server_key: "0e3f833e1ce03b782d9b1392b5fa86be"
server_key_iv: "b7f7cae7c278a632f0a7b3ee0f37ed5a"
server_key_version: "encryptionkey@0"
I20260808 22:26:48.882788 9672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance:
uuid: "ae8e7afb3c91475db96e9811fe0a32d5"
format_stamp: "Formatted at 2026-08-08 22:26:48 on dist-test-slave-7f01"
server_key: "0e3f833e1ce03b782d9b1392b5fa86be"
server_key_iv: "b7f7cae7c278a632f0a7b3ee0f37ed5a"
server_key_version: "encryptionkey@0"
I20260808 22:26:48.888499 9672 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260808 22:26:48.891932 9704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:48.893733 9672 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:26:48.893872 9672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-0/wal
uuid: "ae8e7afb3c91475db96e9811fe0a32d5"
format_stamp: "Formatted at 2026-08-08 22:26:48 on dist-test-slave-7f01"
server_key: "0e3f833e1ce03b782d9b1392b5fa86be"
server_key_iv: "b7f7cae7c278a632f0a7b3ee0f37ed5a"
server_key_version: "encryptionkey@0"
I20260808 22:26:48.894019 9672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:26:48.918164 9672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:26:48.921547 9672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:26:48.921710 9672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:26:48.922276 9672 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:26:48.923091 9672 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:26:48.923139 9672 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:48.923179 9672 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:26:48.923202 9672 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:48.932119 9672 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.193:38401
I20260808 22:26:48.932193 9894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.193:38401 every 8 connection(s)
I20260808 22:26:48.933028 9672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
Aug 08 22:26:48 dist-test-slave-7f01 krb5kdc[9477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228008, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:26:48.942517 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 9672
I20260808 22:26:48.942632 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance
I20260808 22:26:48.943195 5167 external_mini_cluster.cc:1428] Setting key 2415a91436ca115207b139b89fd0ac94
I20260808 22:26:48.947139 9638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:48.934913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:54811 (local address 127.5.11.254:46445)
0808 22:26:48.935046 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:48.935050 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:48.935863 (+ 813us) server_negotiation.cc:407] Connection header received
0808 22:26:48.937237 (+ 1374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:48.937241 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:48.937305 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:48.937412 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:48.938582 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:48.939144 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:48.940442 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:48.940684 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:48.940794 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:48.943478 (+ 2684us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:48.943482 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:48.943484 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:48.943509 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:48.945059 (+ 1550us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:48.945592 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:48.945596 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:48.945598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:48.945682 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:48.946027 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:48.946031 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:48.946033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:48.946235 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:48.946350 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:48.946846 (+ 496us) server_negotiation.cc:299] Negotiation successful
0808 22:26:48.946972 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 22:26:48.948010 9899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:48.935223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46445 (local address 127.5.11.193:54811)
0808 22:26:48.935664 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:48.935707 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:48.937015 (+ 1308us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:48.937602 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:48.937609 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:48.937944 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:48.938430 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:48.938444 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:48.939305 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:48.939308 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:48.940294 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:48.940305 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:48.940847 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:48.940851 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:48.941024 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:48.941547 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:26:48.941562 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:48.943230 (+ 1668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:48.945187 (+ 1957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:48.945193 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:48.945207 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:48.945485 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:48.945777 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:48.945780 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:48.945782 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:48.945929 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:48.946345 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:48.946351 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:48.946594 (+ 243us) client_negotiation.cc:769] Sending connection context
0808 22:26:48.946802 (+ 208us) client_negotiation.cc:240] Negotiation successful
0808 22:26:48.947128 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":124,"threads_started":1}
I20260808 22:26:48.949241 9895 heartbeater.cc:347] Connected to a master server at 127.5.11.254:46445
I20260808 22:26:48.949498 9895 heartbeater.cc:464] Registering TS with master...
I20260808 22:26:48.950148 9895 heartbeater.cc:511] Master 127.5.11.254:46445 requested a full tablet report, sending...
I20260808 22:26:48.951710 9560 ts_manager.cc:194] Registered new tserver with Master: ae8e7afb3c91475db96e9811fe0a32d5 (127.5.11.193:38401)
I20260808 22:26:48.952889 9560 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.193@KRBTEST.COM'} at 127.5.11.193:54811
WARNING: no policy specified for kudu/127.5.11.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab.
Entry for principal kudu/127.5.11.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab.
I20260808 22:26:48.972237 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.194:0
--local_ip_for_outbound_sockets=127.5.11.194
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:46445
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:45463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:26:49.086829 9903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:26:49.087080 9903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:26:49.087121 9903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:26:49.091244 9903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:26:49.091302 9903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:26:49.091358 9903 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.194
I20260808 22:26:49.096668 9903 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:45463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:46445
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9903
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.194
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:26:49.097719 9903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:26:49.098899 9903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:26:49.108611 9910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:49.108620 9918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:49.108611 9912 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:49.109009 9903 server_base.cc:1034] running on GCE node
I20260808 22:26:49.109414 9903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:26:49.110051 9903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:26:49.111241 9903 hybrid_clock.cc:648] HybridClock initialized: now 1786228009111230 us; error 28 us; skew 500 ppm
Aug 08 22:26:49 dist-test-slave-7f01 krb5kdc[9477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228009, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:26:49.113988 9903 init.cc:377] Logged in from keytab as kudu/127.5.11.194@KRBTEST.COM (short username kudu)
I20260808 22:26:49.114889 9903 webserver.cc:460] Webserver started at http://127.5.11.194:42497/ using document root <none> and password file <none>
I20260808 22:26:49.115464 9903 fs_manager.cc:359] Metadata directory not provided
I20260808 22:26:49.115564 9903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:26:49.115868 9903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:26:49.117915 9903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-1/data/instance:
uuid: "1c0573d3d6334529909c8fe7df9bdb29"
format_stamp: "Formatted at 2026-08-08 22:26:49 on dist-test-slave-7f01"
server_key: "6d088a595af9916f252155a061274bcc"
server_key_iv: "84a6a2845a075cbf5bac7ae5e8bdefb3"
server_key_version: "encryptionkey@0"
I20260808 22:26:49.118571 9903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance:
uuid: "1c0573d3d6334529909c8fe7df9bdb29"
format_stamp: "Formatted at 2026-08-08 22:26:49 on dist-test-slave-7f01"
server_key: "6d088a595af9916f252155a061274bcc"
server_key_iv: "84a6a2845a075cbf5bac7ae5e8bdefb3"
server_key_version: "encryptionkey@0"
I20260808 22:26:49.122407 9903 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 22:26:49.124867 9930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:49.126009 9903 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:26:49.126152 9903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-1/wal
uuid: "1c0573d3d6334529909c8fe7df9bdb29"
format_stamp: "Formatted at 2026-08-08 22:26:49 on dist-test-slave-7f01"
server_key: "6d088a595af9916f252155a061274bcc"
server_key_iv: "84a6a2845a075cbf5bac7ae5e8bdefb3"
server_key_version: "encryptionkey@0"
I20260808 22:26:49.126264 9903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:26:49.141120 9903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:26:49.143817 9903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:26:49.144004 9903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:26:49.144555 9903 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:26:49.145403 9903 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:26:49.145496 9903 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:49.145596 9903 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:26:49.145648 9903 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:49.154908 9903 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.194:39129
I20260808 22:26:49.154940 10153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.194:39129 every 8 connection(s)
I20260808 22:26:49.155911 9903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
I20260808 22:26:49.159205 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 9903
I20260808 22:26:49.159332 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance
I20260808 22:26:49.159684 5167 external_mini_cluster.cc:1428] Setting key 4722a07370d3bb450f0b7f8a4b0d61e6
Aug 08 22:26:49 dist-test-slave-7f01 krb5kdc[9477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228009, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.11.195@KRBTEST.COM; defaulting to no policy
I20260808 22:26:49.171010 9638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:49.157901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:34475 (local address 127.5.11.254:46445)
0808 22:26:49.158041 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:49.158044 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:49.158861 (+ 817us) server_negotiation.cc:407] Connection header received
0808 22:26:49.160614 (+ 1753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:49.160618 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:49.160664 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:49.160751 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:49.162328 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:49.162834 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:49.164317 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:49.164543 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:49.164650 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:49.167389 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:49.167393 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:49.167395 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:49.167419 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:49.168952 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:49.169473 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:49.169476 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:49.169478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:49.169569 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:49.169919 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:49.169923 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:49.169925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:49.170111 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:49.170224 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:49.170705 (+ 481us) server_negotiation.cc:299] Negotiation successful
0808 22:26:49.170834 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 22:26:49.171960 10157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:49.158274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46445 (local address 127.5.11.194:34475)
0808 22:26:49.158712 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:49.158753 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:49.160369 (+ 1616us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:49.160909 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:49.160915 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:49.161433 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:49.162163 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:49.162188 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:49.162968 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:49.162974 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:49.164179 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:49.164190 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:49.164699 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:49.164703 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:49.164871 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:49.165477 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:26:49.165493 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:49.167214 (+ 1721us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:49.169081 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:49.169088 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:49.169101 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:49.169363 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:49.169667 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:49.169670 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:49.169672 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:49.169818 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:49.170224 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:49.170233 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:49.170466 (+ 233us) client_negotiation.cc:769] Sending connection context
0808 22:26:49.170657 (+ 191us) client_negotiation.cc:240] Negotiation successful
0808 22:26:49.170965 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":114,"threads_started":1}
I20260808 22:26:49.173137 10154 heartbeater.cc:347] Connected to a master server at 127.5.11.254:46445
I20260808 22:26:49.173388 10154 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.195@KRBTEST.COM" created.
I20260808 22:26:49.173902 10154 heartbeater.cc:511] Master 127.5.11.254:46445 requested a full tablet report, sending...
I20260808 22:26:49.174999 9560 ts_manager.cc:194] Registered new tserver with Master: 1c0573d3d6334529909c8fe7df9bdb29 (127.5.11.194:39129)
I20260808 22:26:49.175731 9560 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.194@KRBTEST.COM'} at 127.5.11.194:34475
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab.
Entry for principal kudu/127.5.11.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab.
I20260808 22:26:49.190016 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.195:0
--local_ip_for_outbound_sockets=127.5.11.195
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:46445
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:45463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:26:49.316839 10162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:26:49.317233 10162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:26:49.317334 10162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:26:49.325886 10162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:26:49.325977 10162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:26:49.326112 10162 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.195
I20260808 22:26:49.335969 10162 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:45463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:46445
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.195
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:26:49.337291 10162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:26:49.338359 10162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:26:49.347786 10175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:49.347786 10176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:49.347836 10182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:49.348137 10162 server_base.cc:1034] running on GCE node
I20260808 22:26:49.348593 10162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:26:49.349146 10162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:26:49.350342 10162 hybrid_clock.cc:648] HybridClock initialized: now 1786228009350326 us; error 25 us; skew 500 ppm
Aug 08 22:26:49 dist-test-slave-7f01 krb5kdc[9477](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228009, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:26:49.353159 10162 init.cc:377] Logged in from keytab as kudu/127.5.11.195@KRBTEST.COM (short username kudu)
I20260808 22:26:49.353992 10162 webserver.cc:460] Webserver started at http://127.5.11.195:40997/ using document root <none> and password file <none>
I20260808 22:26:49.354561 10162 fs_manager.cc:359] Metadata directory not provided
I20260808 22:26:49.354653 10162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:26:49.354941 10162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:26:49.357178 10162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-2/data/instance:
uuid: "3c821e175ad04f7abcbce4a66d513e82"
format_stamp: "Formatted at 2026-08-08 22:26:49 on dist-test-slave-7f01"
server_key: "11bad223498e500e656f8ac50603f598"
server_key_iv: "5e5a3b1fd2755bb44a2d3b139dbf6779"
server_key_version: "encryptionkey@0"
I20260808 22:26:49.357815 10162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance:
uuid: "3c821e175ad04f7abcbce4a66d513e82"
format_stamp: "Formatted at 2026-08-08 22:26:49 on dist-test-slave-7f01"
server_key: "11bad223498e500e656f8ac50603f598"
server_key_iv: "5e5a3b1fd2755bb44a2d3b139dbf6779"
server_key_version: "encryptionkey@0"
I20260808 22:26:49.361476 10162 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 22:26:49.363816 10201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:49.364969 10162 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:26:49.365109 10162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-2/wal
uuid: "3c821e175ad04f7abcbce4a66d513e82"
format_stamp: "Formatted at 2026-08-08 22:26:49 on dist-test-slave-7f01"
server_key: "11bad223498e500e656f8ac50603f598"
server_key_iv: "5e5a3b1fd2755bb44a2d3b139dbf6779"
server_key_version: "encryptionkey@0"
I20260808 22:26:49.365228 10162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:26:49.386109 10162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:26:49.389036 10162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:26:49.389240 10162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:26:49.389814 10162 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:26:49.390646 10162 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:26:49.390722 10162 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:49.390794 10162 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:26:49.390846 10162 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:49.399803 10162 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.195:39773
I20260808 22:26:49.399829 10399 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.195:39773 every 8 connection(s)
I20260808 22:26:49.400727 10162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
I20260808 22:26:49.407073 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 10162
I20260808 22:26:49.407202 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance
I20260808 22:26:49.407485 5167 external_mini_cluster.cc:1428] Setting key 3b90f80963a47a244f45a0ef2c29dfb2
Aug 08 22:26:49 dist-test-slave-7f01 krb5kdc[9477](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228009, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:26:49.414424 9638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:49.402557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:55613 (local address 127.5.11.254:46445)
0808 22:26:49.402701 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:49.402705 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:49.403494 (+ 789us) server_negotiation.cc:407] Connection header received
0808 22:26:49.404862 (+ 1368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:49.404866 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:49.404943 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:49.405049 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:49.406237 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:49.406841 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:49.408250 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:49.408479 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:49.408592 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:49.411012 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:49.411016 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:49.411018 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:49.411053 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:49.412395 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:49.412912 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:49.412916 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:49.412918 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:49.413004 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:49.413351 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:49.413355 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:49.413357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:49.413537 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:49.413658 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:49.414137 (+ 479us) server_negotiation.cc:299] Negotiation successful
0808 22:26:49.414252 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 22:26:49.415159 10404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:49.402903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46445 (local address 127.5.11.195:55613)
0808 22:26:49.403345 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:49.403385 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:49.404646 (+ 1261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:49.405224 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:49.405231 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:49.405578 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:49.406090 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:49.406106 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:49.406985 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:49.406991 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:49.408111 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:49.408122 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:49.408629 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:49.408634 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:49.408803 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:49.409380 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:26:49.409402 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:49.410849 (+ 1447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:49.412517 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:49.412523 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:49.412536 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:49.412797 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:49.413101 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:49.413105 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:49.413107 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:49.413251 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:49.413657 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:49.413663 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:49.413890 (+ 227us) client_negotiation.cc:769] Sending connection context
0808 22:26:49.414091 (+ 201us) client_negotiation.cc:240] Negotiation successful
0808 22:26:49.414403 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":132,"threads_started":1}
I20260808 22:26:49.416219 10400 heartbeater.cc:347] Connected to a master server at 127.5.11.254:46445
I20260808 22:26:49.416433 10400 heartbeater.cc:464] Registering TS with master...
I20260808 22:26:49.416846 10400 heartbeater.cc:511] Master 127.5.11.254:46445 requested a full tablet report, sending...
I20260808 22:26:49.417788 9560 ts_manager.cc:194] Registered new tserver with Master: 3c821e175ad04f7abcbce4a66d513e82 (127.5.11.195:39773)
I20260808 22:26:49.418305 9560 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.195@KRBTEST.COM'} at 127.5.11.195:55613
I20260808 22:26:49.421159 5167 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 22:26:49.436198 9638 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:26:49.434777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35550 (local address 127.5.11.254:46445)
0808 22:26:49.434904 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:49.434907 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:49.435107 (+ 200us) server_negotiation.cc:407] Connection header received
0808 22:26:49.435212 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:49.435215 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:49.435253 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:49.435326 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:49.436043 (+ 717us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35550: BlockingRecv error: recv got EOF from 127.0.0.1:35550 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 22:26:49.437449 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 9672
I20260808 22:26:49.464988 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 9903
I20260808 22:26:49.473836 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 10162
I20260808 22:26:49.485654 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 9506
2026-08-08T22:26:49Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2413 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:26:49 dist-test-slave-7f01 krb5kdc[10417](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:26:49 dist-test-slave-7f01 krb5kdc[10417](info): set up 2 sockets
Aug 08 22:26:49 dist-test-slave-7f01 krb5kdc[10417](info): commencing operation
krb5kdc: starting...
W20260808 22:26:50.817047 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.302s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:26:50 dist-test-slave-7f01 krb5kdc[10417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:26:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:26:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.11.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
I20260808 22:26:50.947252 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:41077
--webserver_interface=127.5.11.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:39463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:41077
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:26:51.059226 10443 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:26:51.059522 10443 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:26:51.059613 10443 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:26:51.063594 10443 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:26:51.063675 10443 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:26:51.063714 10443 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:26:51.063766 10443 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:26:51.063822 10443 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:26:51.068954 10443 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:39463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:41077
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:41077
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10443
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:26:51.069939 10443 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:26:51.070919 10443 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:26:51.079846 10456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:51.079838 10450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:51.080004 10443 server_base.cc:1034] running on GCE node
W20260808 22:26:51.079851 10454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:51.080675 10443 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:26:51.081696 10443 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:26:51.082888 10443 hybrid_clock.cc:648] HybridClock initialized: now 1786228011082875 us; error 25 us; skew 500 ppm
Aug 08 22:26:51 dist-test-slave-7f01 krb5kdc[10417](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228011, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:26:51.088677 10443 init.cc:377] Logged in from keytab as kudu/127.5.11.254@KRBTEST.COM (short username kudu)
I20260808 22:26:51.089598 10443 webserver.cc:460] Webserver started at http://127.5.11.254:38111/ using document root <none> and password file <none>
I20260808 22:26:51.090154 10443 fs_manager.cc:359] Metadata directory not provided
I20260808 22:26:51.090248 10443 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:26:51.090487 10443 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:26:51.092573 10443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "03c31d81d5204fb685a3231860bf5d42"
format_stamp: "Formatted at 2026-08-08 22:26:51 on dist-test-slave-7f01"
server_key: "d8e76651b79267bcd5ed0559a3467032"
server_key_iv: "25c605145128c049454cec8733ea66e0"
server_key_version: "encryptionkey@0"
I20260808 22:26:51.093151 10443 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "03c31d81d5204fb685a3231860bf5d42"
format_stamp: "Formatted at 2026-08-08 22:26:51 on dist-test-slave-7f01"
server_key: "d8e76651b79267bcd5ed0559a3467032"
server_key_iv: "25c605145128c049454cec8733ea66e0"
server_key_version: "encryptionkey@0"
I20260808 22:26:51.096853 10443 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 22:26:51.099193 10464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:51.100476 10443 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:26:51.100625 10443 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "03c31d81d5204fb685a3231860bf5d42"
format_stamp: "Formatted at 2026-08-08 22:26:51 on dist-test-slave-7f01"
server_key: "d8e76651b79267bcd5ed0559a3467032"
server_key_iv: "25c605145128c049454cec8733ea66e0"
server_key_version: "encryptionkey@0"
I20260808 22:26:51.100736 10443 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:26:51.112192 10443 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:26:51.114843 10443 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:26:51.115041 10443 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:26:51.122104 10443 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:41077
I20260808 22:26:51.122433 10551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:41077 every 8 connection(s)
I20260808 22:26:51.123060 10443 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:26:51.123363 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 10443
I20260808 22:26:51.123483 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:26:51.123869 5167 external_mini_cluster.cc:1428] Setting key f2cd4c7b9db84d96ffc72f73896c5a18
I20260808 22:26:51.126431 10554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:51.132983 10554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03c31d81d5204fb685a3231860bf5d42: Bootstrap starting.
Aug 08 22:26:51 dist-test-slave-7f01 krb5kdc[10417](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228010, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:26:51.135788 10554 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 03c31d81d5204fb685a3231860bf5d42: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:51.136672 10554 log.cc:824] T 00000000000000000000000000000000 P 03c31d81d5204fb685a3231860bf5d42: Log is configured to *not* fsync() on all Append() calls
I20260808 22:26:51.138931 10554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 03c31d81d5204fb685a3231860bf5d42: No bootstrap required, opened a new log
I20260808 22:26:51.138947 10559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:51.125242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35462 (local address 127.5.11.254:41077)
0808 22:26:51.125665 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:51.125683 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:51.125725 (+ 42us) server_negotiation.cc:407] Connection header received
0808 22:26:51.126577 (+ 852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:51.126600 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:51.127006 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:51.127350 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:51.128138 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:51.129643 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:51.130486 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:51.131161 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:51.131325 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:51.133812 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:51.133818 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:51.133832 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:51.133862 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:51.135861 (+ 1999us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:51.136293 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:51.136299 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:51.136304 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:51.136460 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:51.136732 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:51.136735 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:51.136737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:51.137224 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:51.137368 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:51.137770 (+ 402us) server_negotiation.cc:299] Negotiation successful
0808 22:26:51.138045 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":278,"thread_start_us":112,"threads_started":1}
I20260808 22:26:51.141851 5167 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:26:51.142845 10554 raft_consensus.cc:348] T 00000000000000000000000000000000 P 03c31d81d5204fb685a3231860bf5d42 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03c31d81d5204fb685a3231860bf5d42" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 41077 } }
I20260808 22:26:51.143059 10554 raft_consensus.cc:374] T 00000000000000000000000000000000 P 03c31d81d5204fb685a3231860bf5d42 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:51.143121 10554 raft_consensus.cc:729] T 00000000000000000000000000000000 P 03c31d81d5204fb685a3231860bf5d42 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 03c31d81d5204fb685a3231860bf5d42, State: Initialized, Role: FOLLOWER
I20260808 22:26:51.143560 10554 consensus_queue.cc:260] T 00000000000000000000000000000000 P 03c31d81d5204fb685a3231860bf5d42 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03c31d81d5204fb685a3231860bf5d42" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 41077 } }
I20260808 22:26:51.143697 10554 raft_consensus.cc:388] T 00000000000000000000000000000000 P 03c31d81d5204fb685a3231860bf5d42 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:26:51.143797 10554 raft_consensus.cc:482] T 00000000000000000000000000000000 P 03c31d81d5204fb685a3231860bf5d42 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:26:51.143898 10554 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 03c31d81d5204fb685a3231860bf5d42 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:51.144726 10554 raft_consensus.cc:504] T 00000000000000000000000000000000 P 03c31d81d5204fb685a3231860bf5d42 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03c31d81d5204fb685a3231860bf5d42" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 41077 } }
I20260808 22:26:51.145058 10554 leader_election.cc:302] T 00000000000000000000000000000000 P 03c31d81d5204fb685a3231860bf5d42 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 03c31d81d5204fb685a3231860bf5d42; no voters:
I20260808 22:26:51.145372 10554 leader_election.cc:288] T 00000000000000000000000000000000 P 03c31d81d5204fb685a3231860bf5d42 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:26:51.145447 10564 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 03c31d81d5204fb685a3231860bf5d42 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:26:51.145634 10564 raft_consensus.cc:686] T 00000000000000000000000000000000 P 03c31d81d5204fb685a3231860bf5d42 [term 1 LEADER]: Becoming Leader. State: Replica: 03c31d81d5204fb685a3231860bf5d42, State: Running, Role: LEADER
I20260808 22:26:51.145972 10564 consensus_queue.cc:237] T 00000000000000000000000000000000 P 03c31d81d5204fb685a3231860bf5d42 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03c31d81d5204fb685a3231860bf5d42" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 41077 } }
I20260808 22:26:51.146623 10554 sys_catalog.cc:564] T 00000000000000000000000000000000 P 03c31d81d5204fb685a3231860bf5d42 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:26:51.147696 10569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03c31d81d5204fb685a3231860bf5d42 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 03c31d81d5204fb685a3231860bf5d42. Latest consensus state: current_term: 1 leader_uuid: "03c31d81d5204fb685a3231860bf5d42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03c31d81d5204fb685a3231860bf5d42" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 41077 } } }
I20260808 22:26:51.147871 10569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03c31d81d5204fb685a3231860bf5d42 [sys.catalog]: This master's current role is: LEADER
I20260808 22:26:51.148197 10568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 03c31d81d5204fb685a3231860bf5d42 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "03c31d81d5204fb685a3231860bf5d42" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "03c31d81d5204fb685a3231860bf5d42" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 41077 } } }
I20260808 22:26:51.148299 10568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 03c31d81d5204fb685a3231860bf5d42 [sys.catalog]: This master's current role is: LEADER
I20260808 22:26:51.148414 10578 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:26:51.151881 10578 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:26:51.158219 10578 catalog_manager.cc:1347] Generated new cluster ID: 8a272e4dae43434282ab5fb2e36c6ef6
I20260808 22:26:51.158280 10578 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:26:51.164848 10559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:51.160018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35466 (local address 127.5.11.254:41077)
0808 22:26:51.160168 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:51.160175 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:51.160454 (+ 279us) server_negotiation.cc:407] Connection header received
0808 22:26:51.160618 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:51.160621 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:51.160674 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:51.160758 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:51.161294 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:51.161812 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:51.162685 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:51.162951 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:51.163063 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:51.163984 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:51.163987 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:51.163990 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:26:51.164397 (+ 407us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:51.164426 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:26:51.164443 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:51.164548 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:51.164636 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 22:26:51.164693 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":63}
I20260808 22:26:51.178602 10578 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:26:51.179478 10578 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:26:51.184343 10578 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 03c31d81d5204fb685a3231860bf5d42: Generated new TSK 0
I20260808 22:26:51.185063 10578 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:26:51.194355 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 10443
2026-08-08T22:26:51Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1706 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:26:51 dist-test-slave-7f01 krb5kdc[10611](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:26:51 dist-test-slave-7f01 krb5kdc[10611](info): set up 2 sockets
Aug 08 22:26:51 dist-test-slave-7f01 krb5kdc[10611](info): commencing operation
krb5kdc: starting...
W20260808 22:26:52.448220 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.227s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:26:52 dist-test-slave-7f01 krb5kdc[10611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:26:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:26:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.11.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
I20260808 22:26:52.574517 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:44131
--webserver_interface=127.5.11.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:40037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:44131
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:26:52.689325 10632 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:26:52.689638 10632 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:26:52.689720 10632 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:26:52.693778 10632 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:26:52.693861 10632 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:26:52.693908 10632 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:26:52.693969 10632 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:26:52.694008 10632 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:26:52.699939 10632 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:40037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:44131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:44131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10632
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:26:52.701017 10632 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:26:52.702056 10632 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:26:52.710670 10638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:52.710831 10632 server_base.cc:1034] running on GCE node
W20260808 22:26:52.710672 10642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:52.710670 10639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:52.711580 10632 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:26:52.712714 10632 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:26:52.713912 10632 hybrid_clock.cc:648] HybridClock initialized: now 1786228012713892 us; error 31 us; skew 500 ppm
Aug 08 22:26:52 dist-test-slave-7f01 krb5kdc[10611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228012, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:26:52.716888 10632 init.cc:377] Logged in from keytab as kudu/127.5.11.254@KRBTEST.COM (short username kudu)
I20260808 22:26:52.717788 10632 webserver.cc:460] Webserver started at http://127.5.11.254:40805/ using document root <none> and password file <none>
I20260808 22:26:52.718364 10632 fs_manager.cc:359] Metadata directory not provided
I20260808 22:26:52.718469 10632 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:26:52.718715 10632 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:26:52.720875 10632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "8fd9f3bf93104d0ab49755b1d64c4275"
format_stamp: "Formatted at 2026-08-08 22:26:52 on dist-test-slave-7f01"
server_key: "00d9a5ce430a498b12457709a0b8125c"
server_key_iv: "42cbdf1f84f307091281ee261904821d"
server_key_version: "encryptionkey@0"
I20260808 22:26:52.721472 10632 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "8fd9f3bf93104d0ab49755b1d64c4275"
format_stamp: "Formatted at 2026-08-08 22:26:52 on dist-test-slave-7f01"
server_key: "00d9a5ce430a498b12457709a0b8125c"
server_key_iv: "42cbdf1f84f307091281ee261904821d"
server_key_version: "encryptionkey@0"
I20260808 22:26:52.725314 10632 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260808 22:26:52.727862 10651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:52.729063 10632 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 22:26:52.729211 10632 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "8fd9f3bf93104d0ab49755b1d64c4275"
format_stamp: "Formatted at 2026-08-08 22:26:52 on dist-test-slave-7f01"
server_key: "00d9a5ce430a498b12457709a0b8125c"
server_key_iv: "42cbdf1f84f307091281ee261904821d"
server_key_version: "encryptionkey@0"
I20260808 22:26:52.729324 10632 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:26:52.752242 10632 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:26:52.755584 10632 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:26:52.755838 10632 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:26:52.763286 10632 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:44131
I20260808 22:26:52.763314 10743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:44131 every 8 connection(s)
I20260808 22:26:52.764258 10632 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:26:52.767041 10744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:52.771386 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 10632
I20260808 22:26:52.771484 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:26:52.771803 5167 external_mini_cluster.cc:1428] Setting key 2af38fe4692063a1386f5d238a923876
I20260808 22:26:52.772985 10744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fd9f3bf93104d0ab49755b1d64c4275: Bootstrap starting.
I20260808 22:26:52.775774 10744 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8fd9f3bf93104d0ab49755b1d64c4275: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:52.776662 10744 log.cc:824] T 00000000000000000000000000000000 P 8fd9f3bf93104d0ab49755b1d64c4275: Log is configured to *not* fsync() on all Append() calls
I20260808 22:26:52.778719 10744 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8fd9f3bf93104d0ab49755b1d64c4275: No bootstrap required, opened a new log
Aug 08 22:26:52 dist-test-slave-7f01 krb5kdc[10611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228012, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:26:52.782900 10744 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8fd9f3bf93104d0ab49755b1d64c4275 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fd9f3bf93104d0ab49755b1d64c4275" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 44131 } }
I20260808 22:26:52.783159 10744 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8fd9f3bf93104d0ab49755b1d64c4275 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:52.783233 10744 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8fd9f3bf93104d0ab49755b1d64c4275 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8fd9f3bf93104d0ab49755b1d64c4275, State: Initialized, Role: FOLLOWER
I20260808 22:26:52.783823 10744 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8fd9f3bf93104d0ab49755b1d64c4275 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fd9f3bf93104d0ab49755b1d64c4275" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 44131 } }
I20260808 22:26:52.783993 10744 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8fd9f3bf93104d0ab49755b1d64c4275 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:26:52.784066 10744 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8fd9f3bf93104d0ab49755b1d64c4275 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:26:52.784174 10744 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8fd9f3bf93104d0ab49755b1d64c4275 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:52.785271 10744 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8fd9f3bf93104d0ab49755b1d64c4275 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fd9f3bf93104d0ab49755b1d64c4275" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 44131 } }
I20260808 22:26:52.785637 10744 leader_election.cc:302] T 00000000000000000000000000000000 P 8fd9f3bf93104d0ab49755b1d64c4275 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8fd9f3bf93104d0ab49755b1d64c4275; no voters:
I20260808 22:26:52.785996 10744 leader_election.cc:288] T 00000000000000000000000000000000 P 8fd9f3bf93104d0ab49755b1d64c4275 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:26:52.786091 10751 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8fd9f3bf93104d0ab49755b1d64c4275 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:26:52.786403 10751 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8fd9f3bf93104d0ab49755b1d64c4275 [term 1 LEADER]: Becoming Leader. State: Replica: 8fd9f3bf93104d0ab49755b1d64c4275, State: Running, Role: LEADER
I20260808 22:26:52.786742 10751 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8fd9f3bf93104d0ab49755b1d64c4275 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fd9f3bf93104d0ab49755b1d64c4275" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 44131 } }
I20260808 22:26:52.787400 10744 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8fd9f3bf93104d0ab49755b1d64c4275 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:26:52.787818 10748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:52.773259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33952 (local address 127.5.11.254:44131)
0808 22:26:52.773695 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:52.773715 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:52.773757 (+ 42us) server_negotiation.cc:407] Connection header received
0808 22:26:52.774628 (+ 871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:52.774653 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:52.775072 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:52.775436 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:52.776249 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:52.777612 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:52.778482 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:52.779171 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:52.779346 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:52.782163 (+ 2817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:52.782168 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:52.782181 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:52.782207 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:52.784369 (+ 2162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:52.784793 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:52.784800 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:52.784806 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:52.784967 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:52.785265 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:52.785268 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:52.785270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:52.785771 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:52.786043 (+ 272us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:52.786355 (+ 312us) server_negotiation.cc:299] Negotiation successful
0808 22:26:52.786828 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":123,"threads_started":1}
I20260808 22:26:52.787914 10751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fd9f3bf93104d0ab49755b1d64c4275 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8fd9f3bf93104d0ab49755b1d64c4275" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fd9f3bf93104d0ab49755b1d64c4275" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 44131 } } }
I20260808 22:26:52.788064 10751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fd9f3bf93104d0ab49755b1d64c4275 [sys.catalog]: This master's current role is: LEADER
I20260808 22:26:52.788379 10752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8fd9f3bf93104d0ab49755b1d64c4275 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8fd9f3bf93104d0ab49755b1d64c4275. Latest consensus state: current_term: 1 leader_uuid: "8fd9f3bf93104d0ab49755b1d64c4275" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8fd9f3bf93104d0ab49755b1d64c4275" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 44131 } } }
I20260808 22:26:52.789794 10752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8fd9f3bf93104d0ab49755b1d64c4275 [sys.catalog]: This master's current role is: LEADER
W20260808 22:26:52.792448 10775 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8fd9f3bf93104d0ab49755b1d64c4275: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:26:52.792537 10775 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:26:52.792610 10769 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:26:52.793308 10769 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:26:52.798930 10769 catalog_manager.cc:1347] Generated new cluster ID: cd365a468ffa410d9cff4615bbe94602
I20260808 22:26:52.798996 10769 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:26:52.812002 10769 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:26:52.812908 10769 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:26:52.818478 10769 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8fd9f3bf93104d0ab49755b1d64c4275: Generated new TSK 0
I20260808 22:26:52.819192 10769 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.11.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab.
Entry for principal kudu/127.5.11.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab.
I20260808 22:26:52.848459 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.193:0
--local_ip_for_outbound_sockets=127.5.11.193
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:44131
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:40037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:26:52.966182 10782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:26:52.966440 10782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:26:52.966482 10782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:26:52.970372 10782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:26:52.970427 10782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:26:52.970482 10782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.193
I20260808 22:26:52.975620 10782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:40037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:44131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.193
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:26:52.976676 10782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:26:52.977670 10782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:26:52.987308 10794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:52.987304 10789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:52.987349 10788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:52.987735 10782 server_base.cc:1034] running on GCE node
I20260808 22:26:52.988227 10782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:26:52.988876 10782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:26:52.990091 10782 hybrid_clock.cc:648] HybridClock initialized: now 1786228012990050 us; error 54 us; skew 500 ppm
Aug 08 22:26:52 dist-test-slave-7f01 krb5kdc[10611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228012, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:26:52.993132 10782 init.cc:377] Logged in from keytab as kudu/127.5.11.193@KRBTEST.COM (short username kudu)
I20260808 22:26:52.994055 10782 webserver.cc:460] Webserver started at http://127.5.11.193:42771/ using document root <none> and password file <none>
I20260808 22:26:52.994694 10782 fs_manager.cc:359] Metadata directory not provided
I20260808 22:26:52.994793 10782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:26:52.995054 10782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:26:52.997175 10782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-0/data/instance:
uuid: "ace6a4c07f87484086e4a554acca6faa"
format_stamp: "Formatted at 2026-08-08 22:26:52 on dist-test-slave-7f01"
server_key: "b9bbea74893c9b9ed9ca4c85af6a83a1"
server_key_iv: "c6d65327c8b23caaaf672ce257f8e600"
server_key_version: "encryptionkey@0"
I20260808 22:26:52.997759 10782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance:
uuid: "ace6a4c07f87484086e4a554acca6faa"
format_stamp: "Formatted at 2026-08-08 22:26:52 on dist-test-slave-7f01"
server_key: "b9bbea74893c9b9ed9ca4c85af6a83a1"
server_key_iv: "c6d65327c8b23caaaf672ce257f8e600"
server_key_version: "encryptionkey@0"
I20260808 22:26:53.001572 10782 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 22:26:53.004019 10811 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:53.005234 10782 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:26:53.005378 10782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-0/wal
uuid: "ace6a4c07f87484086e4a554acca6faa"
format_stamp: "Formatted at 2026-08-08 22:26:52 on dist-test-slave-7f01"
server_key: "b9bbea74893c9b9ed9ca4c85af6a83a1"
server_key_iv: "c6d65327c8b23caaaf672ce257f8e600"
server_key_version: "encryptionkey@0"
I20260808 22:26:53.005506 10782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:26:53.021773 10782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:26:53.024693 10782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:26:53.024916 10782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:26:53.025502 10782 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:26:53.026345 10782 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:26:53.026423 10782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:53.026496 10782 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:26:53.026552 10782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:53.036197 10782 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.193:38449
I20260808 22:26:53.036218 10993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.193:38449 every 8 connection(s)
I20260808 22:26:53.037078 10782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
I20260808 22:26:53.045347 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 10782
I20260808 22:26:53.045506 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance
I20260808 22:26:53.045806 5167 external_mini_cluster.cc:1428] Setting key 9391c05ea316b1b4f3e066af8540a98b
Aug 08 22:26:53 dist-test-slave-7f01 krb5kdc[10611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228012, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:26:53.050895 10748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:53.038839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:41537 (local address 127.5.11.254:44131)
0808 22:26:53.038991 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:53.038996 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:53.039917 (+ 921us) server_negotiation.cc:407] Connection header received
0808 22:26:53.041227 (+ 1310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:53.041230 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:53.041278 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:53.041353 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:53.042500 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:53.043032 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:53.044125 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:53.044347 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:53.044439 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:53.047250 (+ 2811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:53.047253 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:53.047255 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:53.047277 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:53.048800 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:53.049338 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:53.049343 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:53.049345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:53.049441 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:53.049793 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:53.049797 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:53.049799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:53.049987 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:53.050099 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:53.050586 (+ 487us) server_negotiation.cc:299] Negotiation successful
0808 22:26:53.050714 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 22:26:53.051689 11001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:53.039272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:44131 (local address 127.5.11.193:41537)
0808 22:26:53.039708 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:53.039805 (+ 97us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:53.041028 (+ 1223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:53.041513 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:53.041519 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:53.041868 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:53.042363 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:53.042377 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:53.043138 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:53.043141 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:53.044021 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:53.044031 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:53.044507 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:53.044510 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:53.044659 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:53.045324 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:26:53.045341 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:53.047067 (+ 1726us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:53.048942 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:53.048948 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:53.048963 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:53.049228 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:53.049539 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:53.049543 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:53.049545 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:53.049693 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:53.050099 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:53.050107 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:53.050349 (+ 242us) client_negotiation.cc:769] Sending connection context
0808 22:26:53.050541 (+ 192us) client_negotiation.cc:240] Negotiation successful
0808 22:26:53.050857 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":119,"threads_started":1}
I20260808 22:26:53.053025 10997 heartbeater.cc:347] Connected to a master server at 127.5.11.254:44131
I20260808 22:26:53.053303 10997 heartbeater.cc:464] Registering TS with master...
I20260808 22:26:53.053952 10997 heartbeater.cc:511] Master 127.5.11.254:44131 requested a full tablet report, sending...
I20260808 22:26:53.055567 10680 ts_manager.cc:194] Registered new tserver with Master: ace6a4c07f87484086e4a554acca6faa (127.5.11.193:38449)
WARNING: no policy specified for kudu/127.5.11.194@KRBTEST.COM; defaulting to no policy
I20260808 22:26:53.056744 10680 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.193@KRBTEST.COM'} at 127.5.11.193:41537
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab.
Entry for principal kudu/127.5.11.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab.
I20260808 22:26:53.074287 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.194:0
--local_ip_for_outbound_sockets=127.5.11.194
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:44131
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:40037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:26:53.187927 11008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:26:53.188169 11008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:26:53.188236 11008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:26:53.192441 11008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:26:53.192497 11008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:26:53.192600 11008 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.194
I20260808 22:26:53.197799 11008 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:40037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:44131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11008
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.194
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:26:53.198813 11008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:26:53.199893 11008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:26:53.209357 11017 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:53.209383 11019 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:53.209357 11016 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:53.209685 11008 server_base.cc:1034] running on GCE node
I20260808 22:26:53.210230 11008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:26:53.210803 11008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:26:53.211978 11008 hybrid_clock.cc:648] HybridClock initialized: now 1786228013211947 us; error 42 us; skew 500 ppm
Aug 08 22:26:53 dist-test-slave-7f01 krb5kdc[10611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228013, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:26:53.214746 11008 init.cc:377] Logged in from keytab as kudu/127.5.11.194@KRBTEST.COM (short username kudu)
I20260808 22:26:53.215687 11008 webserver.cc:460] Webserver started at http://127.5.11.194:33173/ using document root <none> and password file <none>
I20260808 22:26:53.216315 11008 fs_manager.cc:359] Metadata directory not provided
I20260808 22:26:53.216383 11008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:26:53.216634 11008 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:26:53.218705 11008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-1/data/instance:
uuid: "0fd26256a68f4148b77f647438ed76e9"
format_stamp: "Formatted at 2026-08-08 22:26:53 on dist-test-slave-7f01"
server_key: "49c1a14fb901980126b08a87ee0452d9"
server_key_iv: "79ecd001d0f1da1a6baf020dad508b45"
server_key_version: "encryptionkey@0"
I20260808 22:26:53.219296 11008 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance:
uuid: "0fd26256a68f4148b77f647438ed76e9"
format_stamp: "Formatted at 2026-08-08 22:26:53 on dist-test-slave-7f01"
server_key: "49c1a14fb901980126b08a87ee0452d9"
server_key_iv: "79ecd001d0f1da1a6baf020dad508b45"
server_key_version: "encryptionkey@0"
I20260808 22:26:53.223100 11008 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 22:26:53.225618 11032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:53.226886 11008 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 22:26:53.227044 11008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-1/wal
uuid: "0fd26256a68f4148b77f647438ed76e9"
format_stamp: "Formatted at 2026-08-08 22:26:53 on dist-test-slave-7f01"
server_key: "49c1a14fb901980126b08a87ee0452d9"
server_key_iv: "79ecd001d0f1da1a6baf020dad508b45"
server_key_version: "encryptionkey@0"
I20260808 22:26:53.227160 11008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:26:53.238123 11008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:26:53.241060 11008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:26:53.241214 11008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:26:53.241732 11008 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:26:53.242540 11008 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:26:53.242588 11008 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:53.242633 11008 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:26:53.242656 11008 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:53.251986 11008 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.194:34743
I20260808 22:26:53.252010 11242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.194:34743 every 8 connection(s)
I20260808 22:26:53.252880 11008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
I20260808 22:26:53.260826 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 11008
I20260808 22:26:53.260936 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance
I20260808 22:26:53.261251 5167 external_mini_cluster.cc:1428] Setting key 63eb8b65932bb22b0c9aa0adc42e78f3
Aug 08 22:26:53 dist-test-slave-7f01 krb5kdc[10611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228013, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:26:53.267208 10748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:53.254674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:59657 (local address 127.5.11.254:44131)
0808 22:26:53.254816 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:53.254820 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:53.255603 (+ 783us) server_negotiation.cc:407] Connection header received
0808 22:26:53.257051 (+ 1448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:53.257054 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:53.257119 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:53.257223 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:53.258382 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:53.258922 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:53.260236 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:53.260468 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:53.260582 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:53.263318 (+ 2736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:53.263321 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:53.263323 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:53.263346 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:53.265009 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:53.265501 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:53.265507 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:53.265510 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:53.265612 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:53.265955 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:53.265961 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:53.265964 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:53.266215 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:53.266325 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:53.266805 (+ 480us) server_negotiation.cc:299] Negotiation successful
0808 22:26:53.266962 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 22:26:53.267860 11246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:53.254983 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:44131 (local address 127.5.11.194:59657)
0808 22:26:53.255441 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:53.255484 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:53.256814 (+ 1330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:53.257405 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:53.257412 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:53.257757 (+ 345us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:53.258223 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:53.258236 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:53.259064 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:53.259067 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:53.260112 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:53.260122 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:53.260644 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:53.260648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:53.260812 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:53.261522 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:26:53.261542 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:53.263153 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:53.265130 (+ 1977us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:53.265134 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:53.265146 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:53.265402 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:53.265707 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:53.265709 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:53.265711 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:53.265863 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:53.266321 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:53.266326 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:53.266550 (+ 224us) client_negotiation.cc:769] Sending connection context
0808 22:26:53.266699 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 22:26:53.266975 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":119,"threads_started":1}
I20260808 22:26:53.269062 11243 heartbeater.cc:347] Connected to a master server at 127.5.11.254:44131
I20260808 22:26:53.269345 11243 heartbeater.cc:464] Registering TS with master...
I20260808 22:26:53.270001 11243 heartbeater.cc:511] Master 127.5.11.254:44131 requested a full tablet report, sending...
I20260808 22:26:53.271183 10680 ts_manager.cc:194] Registered new tserver with Master: 0fd26256a68f4148b77f647438ed76e9 (127.5.11.194:34743)
I20260808 22:26:53.271838 10680 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.194@KRBTEST.COM'} at 127.5.11.194:59657
WARNING: no policy specified for kudu/127.5.11.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab.
Entry for principal kudu/127.5.11.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab.
I20260808 22:26:53.291127 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.195:0
--local_ip_for_outbound_sockets=127.5.11.195
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:44131
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:40037
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:26:53.410725 11259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:26:53.411010 11259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:26:53.411092 11259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:26:53.415069 11259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:26:53.415154 11259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:26:53.415237 11259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.195
I20260808 22:26:53.420369 11259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:40037
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:44131
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11259
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.195
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:26:53.421396 11259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:26:53.422417 11259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:26:53.432062 11270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:53.432057 11273 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:53.432057 11267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:53.432372 11259 server_base.cc:1034] running on GCE node
I20260808 22:26:53.432809 11259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:26:53.433501 11259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:26:53.434689 11259 hybrid_clock.cc:648] HybridClock initialized: now 1786228013434667 us; error 34 us; skew 500 ppm
Aug 08 22:26:53 dist-test-slave-7f01 krb5kdc[10611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228013, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:26:53.437584 11259 init.cc:377] Logged in from keytab as kudu/127.5.11.195@KRBTEST.COM (short username kudu)
I20260808 22:26:53.438436 11259 webserver.cc:460] Webserver started at http://127.5.11.195:34903/ using document root <none> and password file <none>
I20260808 22:26:53.439007 11259 fs_manager.cc:359] Metadata directory not provided
I20260808 22:26:53.439105 11259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:26:53.439347 11259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:26:53.441447 11259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-2/data/instance:
uuid: "f3381562a3ff4f32bbbc511c56ce9fd5"
format_stamp: "Formatted at 2026-08-08 22:26:53 on dist-test-slave-7f01"
server_key: "c37686a736d5356da051d8c66636a8a7"
server_key_iv: "8a88a99f81ac69a78be423ceb59467ac"
server_key_version: "encryptionkey@0"
I20260808 22:26:53.442032 11259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance:
uuid: "f3381562a3ff4f32bbbc511c56ce9fd5"
format_stamp: "Formatted at 2026-08-08 22:26:53 on dist-test-slave-7f01"
server_key: "c37686a736d5356da051d8c66636a8a7"
server_key_iv: "8a88a99f81ac69a78be423ceb59467ac"
server_key_version: "encryptionkey@0"
I20260808 22:26:53.445799 11259 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260808 22:26:53.448226 11285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:53.449471 11259 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:26:53.449611 11259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-2/wal
uuid: "f3381562a3ff4f32bbbc511c56ce9fd5"
format_stamp: "Formatted at 2026-08-08 22:26:53 on dist-test-slave-7f01"
server_key: "c37686a736d5356da051d8c66636a8a7"
server_key_iv: "8a88a99f81ac69a78be423ceb59467ac"
server_key_version: "encryptionkey@0"
I20260808 22:26:53.449725 11259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:26:53.472810 11259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:26:53.476282 11259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:26:53.476505 11259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:26:53.477119 11259 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:26:53.477982 11259 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:26:53.478060 11259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:53.478135 11259 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:26:53.478181 11259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:53.487285 11259 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.195:33191
I20260808 22:26:53.487300 11482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.195:33191 every 8 connection(s)
I20260808 22:26:53.488330 11259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
I20260808 22:26:53.497928 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 11259
I20260808 22:26:53.498067 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance
Aug 08 22:26:53 dist-test-slave-7f01 krb5kdc[10611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228013, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:26:53.498401 5167 external_mini_cluster.cc:1428] Setting key e95cac8d1cff1f478a7bf2ec4c1c828d
I20260808 22:26:53.502758 10748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:53.490304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:51109 (local address 127.5.11.254:44131)
0808 22:26:53.490456 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:53.490460 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:53.491291 (+ 831us) server_negotiation.cc:407] Connection header received
0808 22:26:53.492784 (+ 1493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:53.492788 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:53.492850 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:53.492963 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:53.494146 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:53.494690 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:53.495857 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:53.496154 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:53.496263 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:53.498948 (+ 2685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:53.498954 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:53.498960 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:53.498990 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:53.500923 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:53.501448 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:53.501451 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:53.501452 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:53.501531 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:53.501841 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:53.501844 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:53.501845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:53.502013 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:53.502090 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:53.502537 (+ 447us) server_negotiation.cc:299] Negotiation successful
0808 22:26:53.502636 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 22:26:53.503611 11486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:53.490684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:44131 (local address 127.5.11.195:51109)
0808 22:26:53.491140 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:53.491182 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:53.492568 (+ 1386us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:53.493149 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:53.493156 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:53.493512 (+ 356us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:53.493994 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:53.494008 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:53.494824 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:53.494833 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:53.495696 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:53.495706 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:53.496316 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:53.496320 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:53.496491 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:53.497018 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:26:53.497035 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:53.498782 (+ 1747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:53.501051 (+ 2269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:53.501060 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:53.501075 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:53.501356 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:53.501617 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:53.501619 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:53.501621 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:53.501758 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:53.502103 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:53.502109 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:53.502328 (+ 219us) client_negotiation.cc:769] Sending connection context
0808 22:26:53.502482 (+ 154us) client_negotiation.cc:240] Negotiation successful
0808 22:26:53.502736 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":113,"threads_started":1}
I20260808 22:26:53.504817 11483 heartbeater.cc:347] Connected to a master server at 127.5.11.254:44131
I20260808 22:26:53.505029 11483 heartbeater.cc:464] Registering TS with master...
I20260808 22:26:53.505532 11483 heartbeater.cc:511] Master 127.5.11.254:44131 requested a full tablet report, sending...
I20260808 22:26:53.506502 10680 ts_manager.cc:194] Registered new tserver with Master: f3381562a3ff4f32bbbc511c56ce9fd5 (127.5.11.195:33191)
I20260808 22:26:53.507120 10680 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.195@KRBTEST.COM'} at 127.5.11.195:51109
I20260808 22:26:53.512584 5167 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 22:26:53 dist-test-slave-7f01 krb5kdc[10611](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228013, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 08 22:26:53 dist-test-slave-7f01 krb5kdc[10611](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228013, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:26:53.539865 10748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:53.531158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33968 (local address 127.5.11.254:44131)
0808 22:26:53.531335 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:53.531340 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:53.531602 (+ 262us) server_negotiation.cc:407] Connection header received
0808 22:26:53.531717 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:53.531720 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:53.531812 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:53.531884 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:53.532705 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:53.533243 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:53.534120 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:53.534336 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:53.534412 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:53.536864 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:53.536867 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:53.536869 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:53.536900 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:53.538250 (+ 1350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:53.538651 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:53.538654 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:53.538655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:53.538733 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:53.538968 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:53.538970 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:53.538971 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:53.539195 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:53.539300 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:53.539497 (+ 197us) server_negotiation.cc:299] Negotiation successful
0808 22:26:53.539607 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260808 22:26:53.540314 10680 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:33968
I20260808 22:26:53.541203 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 10782
I20260808 22:26:53.562525 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 11008
I20260808 22:26:53.571266 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 11259
I20260808 22:26:53.580747 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 10632
2026-08-08T22:26:53Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2387 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:26:53 dist-test-slave-7f01 krb5kdc[11508](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:26:53 dist-test-slave-7f01 krb5kdc[11508](info): set up 2 sockets
Aug 08 22:26:53 dist-test-slave-7f01 krb5kdc[11508](info): commencing operation
krb5kdc: starting...
W20260808 22:26:54.923933 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.315s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:26:55 dist-test-slave-7f01 krb5kdc[11508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:26:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:26:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.11.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
I20260808 22:26:55.054400 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:41873
--webserver_interface=127.5.11.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:34309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:41873
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:26:55.168825 11526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:26:55.169080 11526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:26:55.169118 11526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:26:55.172976 11526 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:26:55.173025 11526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:26:55.173044 11526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:26:55.173060 11526 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:26:55.173075 11526 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:26:55.178161 11526 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:34309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:41873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:41873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:26:55.179093 11526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:26:55.180087 11526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:26:55.188395 11536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:55.188402 11539 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:55.188412 11534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:55.188653 11526 server_base.cc:1034] running on GCE node
I20260808 22:26:55.189318 11526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:26:55.190383 11526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:26:55.191582 11526 hybrid_clock.cc:648] HybridClock initialized: now 1786228015191559 us; error 40 us; skew 500 ppm
Aug 08 22:26:55 dist-test-slave-7f01 krb5kdc[11508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228015, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:26:55.194312 11526 init.cc:377] Logged in from keytab as kudu/127.5.11.254@KRBTEST.COM (short username kudu)
I20260808 22:26:55.195160 11526 webserver.cc:460] Webserver started at http://127.5.11.254:39963/ using document root <none> and password file <none>
I20260808 22:26:55.195812 11526 fs_manager.cc:359] Metadata directory not provided
I20260808 22:26:55.195905 11526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:26:55.196125 11526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:26:55.198156 11526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "e449484bfe054f1896911335080b963d"
format_stamp: "Formatted at 2026-08-08 22:26:55 on dist-test-slave-7f01"
server_key: "f1a17589b9c30429d7ecf4f1f34fefaf"
server_key_iv: "463e1e628a0e88b62d2067511caaeb6f"
server_key_version: "encryptionkey@0"
I20260808 22:26:55.198742 11526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "e449484bfe054f1896911335080b963d"
format_stamp: "Formatted at 2026-08-08 22:26:55 on dist-test-slave-7f01"
server_key: "f1a17589b9c30429d7ecf4f1f34fefaf"
server_key_iv: "463e1e628a0e88b62d2067511caaeb6f"
server_key_version: "encryptionkey@0"
I20260808 22:26:55.202441 11526 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 22:26:55.204826 11551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:55.205958 11526 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:26:55.206102 11526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "e449484bfe054f1896911335080b963d"
format_stamp: "Formatted at 2026-08-08 22:26:55 on dist-test-slave-7f01"
server_key: "f1a17589b9c30429d7ecf4f1f34fefaf"
server_key_iv: "463e1e628a0e88b62d2067511caaeb6f"
server_key_version: "encryptionkey@0"
I20260808 22:26:55.206214 11526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:26:55.222606 11526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:26:55.225592 11526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:26:55.225800 11526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:26:55.232987 11526 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:41873
I20260808 22:26:55.233028 11640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:41873 every 8 connection(s)
I20260808 22:26:55.233937 11526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:26:55.236733 11641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:55.240705 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 11526
I20260808 22:26:55.240808 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:26:55.241119 5167 external_mini_cluster.cc:1428] Setting key db8b5fa393e92e03fdc6dedbd965c585
I20260808 22:26:55.242659 11641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e449484bfe054f1896911335080b963d: Bootstrap starting.
I20260808 22:26:55.245500 11641 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e449484bfe054f1896911335080b963d: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:55.246407 11641 log.cc:824] T 00000000000000000000000000000000 P e449484bfe054f1896911335080b963d: Log is configured to *not* fsync() on all Append() calls
I20260808 22:26:55.248512 11641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e449484bfe054f1896911335080b963d: No bootstrap required, opened a new log
Aug 08 22:26:55 dist-test-slave-7f01 krb5kdc[11508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:26:55.252813 11641 raft_consensus.cc:348] T 00000000000000000000000000000000 P e449484bfe054f1896911335080b963d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e449484bfe054f1896911335080b963d" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 41873 } }
I20260808 22:26:55.253098 11641 raft_consensus.cc:374] T 00000000000000000000000000000000 P e449484bfe054f1896911335080b963d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:55.253165 11641 raft_consensus.cc:729] T 00000000000000000000000000000000 P e449484bfe054f1896911335080b963d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e449484bfe054f1896911335080b963d, State: Initialized, Role: FOLLOWER
I20260808 22:26:55.253643 11641 consensus_queue.cc:260] T 00000000000000000000000000000000 P e449484bfe054f1896911335080b963d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e449484bfe054f1896911335080b963d" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 41873 } }
I20260808 22:26:55.253809 11641 raft_consensus.cc:388] T 00000000000000000000000000000000 P e449484bfe054f1896911335080b963d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:26:55.253886 11641 raft_consensus.cc:482] T 00000000000000000000000000000000 P e449484bfe054f1896911335080b963d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:26:55.253988 11641 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e449484bfe054f1896911335080b963d [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:55.255126 11641 raft_consensus.cc:504] T 00000000000000000000000000000000 P e449484bfe054f1896911335080b963d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e449484bfe054f1896911335080b963d" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 41873 } }
I20260808 22:26:55.255494 11641 leader_election.cc:302] T 00000000000000000000000000000000 P e449484bfe054f1896911335080b963d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e449484bfe054f1896911335080b963d; no voters:
I20260808 22:26:55.255846 11641 leader_election.cc:288] T 00000000000000000000000000000000 P e449484bfe054f1896911335080b963d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:26:55.255955 11652 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e449484bfe054f1896911335080b963d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:26:55.256194 11652 raft_consensus.cc:686] T 00000000000000000000000000000000 P e449484bfe054f1896911335080b963d [term 1 LEADER]: Becoming Leader. State: Replica: e449484bfe054f1896911335080b963d, State: Running, Role: LEADER
I20260808 22:26:55.256506 11652 consensus_queue.cc:237] T 00000000000000000000000000000000 P e449484bfe054f1896911335080b963d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e449484bfe054f1896911335080b963d" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 41873 } }
I20260808 22:26:55.256736 11644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:55.242592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43066 (local address 127.5.11.254:41873)
0808 22:26:55.243028 (+ 436us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:55.243048 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:55.243091 (+ 43us) server_negotiation.cc:407] Connection header received
0808 22:26:55.243966 (+ 875us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:55.243991 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:55.244414 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:55.244857 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:55.245526 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:55.247015 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:55.247955 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:55.248644 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:55.248805 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:55.251216 (+ 2411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:55.251225 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:55.251247 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:55.251286 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:55.253358 (+ 2072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:55.253813 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:55.253822 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:55.253832 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:55.253985 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:55.254313 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:55.254318 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:55.254319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:55.254836 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:55.255070 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:55.255335 (+ 265us) server_negotiation.cc:299] Negotiation successful
0808 22:26:55.255641 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":132,"threads_started":1}
I20260808 22:26:55.257130 11641 sys_catalog.cc:564] T 00000000000000000000000000000000 P e449484bfe054f1896911335080b963d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:26:55.257937 11656 sys_catalog.cc:455] T 00000000000000000000000000000000 P e449484bfe054f1896911335080b963d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e449484bfe054f1896911335080b963d. Latest consensus state: current_term: 1 leader_uuid: "e449484bfe054f1896911335080b963d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e449484bfe054f1896911335080b963d" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 41873 } } }
I20260808 22:26:55.258059 11656 sys_catalog.cc:458] T 00000000000000000000000000000000 P e449484bfe054f1896911335080b963d [sys.catalog]: This master's current role is: LEADER
I20260808 22:26:55.258762 11653 sys_catalog.cc:455] T 00000000000000000000000000000000 P e449484bfe054f1896911335080b963d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e449484bfe054f1896911335080b963d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e449484bfe054f1896911335080b963d" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 41873 } } }
I20260808 22:26:55.258934 11653 sys_catalog.cc:458] T 00000000000000000000000000000000 P e449484bfe054f1896911335080b963d [sys.catalog]: This master's current role is: LEADER
W20260808 22:26:55.261858 11676 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e449484bfe054f1896911335080b963d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:26:55.261952 11676 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:26:55.262053 11668 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:26:55.262931 11668 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:26:55.270403 11668 catalog_manager.cc:1347] Generated new cluster ID: a4b7ed83bd654e6287883b970197a09e
I20260808 22:26:55.270468 11668 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:26:55.297780 11668 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:26:55.299361 11668 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:26:55.309383 11668 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e449484bfe054f1896911335080b963d: Generated new TSK 0
I20260808 22:26:55.310194 11668 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.11.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab.
Entry for principal kudu/127.5.11.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab.
I20260808 22:26:55.351141 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.193:0
--local_ip_for_outbound_sockets=127.5.11.193
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:41873
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:34309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:26:55.500313 11687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:26:55.500613 11687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:26:55.500715 11687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:26:55.504765 11687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:26:55.504846 11687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:26:55.504936 11687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.193
I20260808 22:26:55.510320 11687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:34309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:41873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.193
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:26:55.511503 11687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:26:55.512631 11687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:26:55.522536 11692 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:55.522838 11687 server_base.cc:1034] running on GCE node
W20260808 22:26:55.522641 11696 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:55.522573 11693 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:55.523531 11687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:26:55.524237 11687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:26:55.525435 11687 hybrid_clock.cc:648] HybridClock initialized: now 1786228015525405 us; error 53 us; skew 500 ppm
Aug 08 22:26:55 dist-test-slave-7f01 krb5kdc[11508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228015, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:26:55.528574 11687 init.cc:377] Logged in from keytab as kudu/127.5.11.193@KRBTEST.COM (short username kudu)
I20260808 22:26:55.529536 11687 webserver.cc:460] Webserver started at http://127.5.11.193:39555/ using document root <none> and password file <none>
I20260808 22:26:55.530154 11687 fs_manager.cc:359] Metadata directory not provided
I20260808 22:26:55.530252 11687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:26:55.530503 11687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:26:55.532748 11687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-0/data/instance:
uuid: "4374eabdd9814b8f9abaca278edf0c3e"
format_stamp: "Formatted at 2026-08-08 22:26:55 on dist-test-slave-7f01"
server_key: "baecff0880551505342aa1c09438332b"
server_key_iv: "649832a1e48ff7fac4c9817f2f075801"
server_key_version: "encryptionkey@0"
I20260808 22:26:55.533377 11687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance:
uuid: "4374eabdd9814b8f9abaca278edf0c3e"
format_stamp: "Formatted at 2026-08-08 22:26:55 on dist-test-slave-7f01"
server_key: "baecff0880551505342aa1c09438332b"
server_key_iv: "649832a1e48ff7fac4c9817f2f075801"
server_key_version: "encryptionkey@0"
I20260808 22:26:55.537446 11687 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 22:26:55.540015 11708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:55.541326 11687 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:26:55.541426 11687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-0/wal
uuid: "4374eabdd9814b8f9abaca278edf0c3e"
format_stamp: "Formatted at 2026-08-08 22:26:55 on dist-test-slave-7f01"
server_key: "baecff0880551505342aa1c09438332b"
server_key_iv: "649832a1e48ff7fac4c9817f2f075801"
server_key_version: "encryptionkey@0"
I20260808 22:26:55.541561 11687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:26:55.558949 11687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:26:55.562331 11687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:26:55.562546 11687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:26:55.563297 11687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:26:55.564291 11687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:26:55.564344 11687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:55.564406 11687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:26:55.564438 11687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:55.576347 11687 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.193:44857
I20260808 22:26:55.576344 11925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.193:44857 every 8 connection(s)
I20260808 22:26:55.577749 11687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
I20260808 22:26:55.579115 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 11687
I20260808 22:26:55.579253 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance
I20260808 22:26:55.579583 5167 external_mini_cluster.cc:1428] Setting key 90c6d522aa7f3f2f1e008beabe121901
Aug 08 22:26:55 dist-test-slave-7f01 krb5kdc[11508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228015, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.11.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.194@KRBTEST.COM" created.
I20260808 22:26:55.596176 11644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:55.582029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:56487 (local address 127.5.11.254:41873)
0808 22:26:55.582188 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:55.582193 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:55.582209 (+ 16us) server_negotiation.cc:407] Connection header received
0808 22:26:55.583689 (+ 1480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:55.583694 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:55.583806 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:55.583916 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:55.585637 (+ 1721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:55.586321 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:55.588288 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:55.588776 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:55.588919 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:55.592238 (+ 3319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:55.592241 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:55.592244 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:55.592270 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:55.593798 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:55.594374 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:55.594378 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:55.594380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:55.594525 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:55.594994 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:55.595001 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:55.595003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:55.595205 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:55.595321 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:55.595870 (+ 549us) server_negotiation.cc:299] Negotiation successful
0808 22:26:55.596003 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 22:26:55.597117 11929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:55.580787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:41873 (local address 127.5.11.193:56487)
0808 22:26:55.581454 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:55.581517 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:55.583446 (+ 1929us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:55.584151 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:55.584160 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:55.584699 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:55.585459 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:55.585484 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:55.586471 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:55.586478 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:55.588130 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:55.588147 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:55.588982 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:55.588989 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:55.589228 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:55.590028 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:26:55.590052 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:55.592062 (+ 2010us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:55.593935 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:55.593940 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:55.593953 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:55.594226 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:55.594663 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:55.594668 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:55.594670 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:55.594889 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:55.595320 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:55.595327 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:55.595596 (+ 269us) client_negotiation.cc:769] Sending connection context
0808 22:26:55.595818 (+ 222us) client_negotiation.cc:240] Negotiation successful
0808 22:26:55.596157 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":196,"threads_started":1}
I20260808 22:26:55.598405 11926 heartbeater.cc:347] Connected to a master server at 127.5.11.254:41873
I20260808 22:26:55.598682 11926 heartbeater.cc:464] Registering TS with master...
I20260808 22:26:55.599363 11926 heartbeater.cc:511] Master 127.5.11.254:41873 requested a full tablet report, sending...
I20260808 22:26:55.601083 11585 ts_manager.cc:194] Registered new tserver with Master: 4374eabdd9814b8f9abaca278edf0c3e (127.5.11.193:44857)
I20260808 22:26:55.602240 11585 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.193@KRBTEST.COM'} at 127.5.11.193:56487
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab.
Entry for principal kudu/127.5.11.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab.
I20260808 22:26:55.611315 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.194:0
--local_ip_for_outbound_sockets=127.5.11.194
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:41873
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:34309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:26:55.719830 11935 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:26:55.720101 11935 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:26:55.720178 11935 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:26:55.724098 11935 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:26:55.724182 11935 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:26:55.724272 11935 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.194
I20260808 22:26:55.729264 11935 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:34309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:41873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11935
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.194
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:26:55.730721 11935 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:26:55.731720 11935 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:26:55.740790 11940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:55.740818 11945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:55.740829 11942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:55.741238 11935 server_base.cc:1034] running on GCE node
I20260808 22:26:55.741636 11935 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:26:55.742231 11935 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:26:55.743420 11935 hybrid_clock.cc:648] HybridClock initialized: now 1786228015743403 us; error 32 us; skew 500 ppm
Aug 08 22:26:55 dist-test-slave-7f01 krb5kdc[11508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228015, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:26:55.746411 11935 init.cc:377] Logged in from keytab as kudu/127.5.11.194@KRBTEST.COM (short username kudu)
I20260808 22:26:55.747385 11935 webserver.cc:460] Webserver started at http://127.5.11.194:43409/ using document root <none> and password file <none>
I20260808 22:26:55.748040 11935 fs_manager.cc:359] Metadata directory not provided
I20260808 22:26:55.748136 11935 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:26:55.748366 11935 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:26:55.750334 11935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-1/data/instance:
uuid: "f6a89330010b48ae8bd11523b4a9779a"
format_stamp: "Formatted at 2026-08-08 22:26:55 on dist-test-slave-7f01"
server_key: "8b5382c9a8ec2beac727e4b2e6ba0416"
server_key_iv: "be8a51b070b55055ef3e31ab9862fef7"
server_key_version: "encryptionkey@0"
I20260808 22:26:55.750903 11935 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance:
uuid: "f6a89330010b48ae8bd11523b4a9779a"
format_stamp: "Formatted at 2026-08-08 22:26:55 on dist-test-slave-7f01"
server_key: "8b5382c9a8ec2beac727e4b2e6ba0416"
server_key_iv: "be8a51b070b55055ef3e31ab9862fef7"
server_key_version: "encryptionkey@0"
I20260808 22:26:55.754534 11935 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260808 22:26:55.756970 11955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:55.758277 11935 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:26:55.758412 11935 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-1/wal
uuid: "f6a89330010b48ae8bd11523b4a9779a"
format_stamp: "Formatted at 2026-08-08 22:26:55 on dist-test-slave-7f01"
server_key: "8b5382c9a8ec2beac727e4b2e6ba0416"
server_key_iv: "be8a51b070b55055ef3e31ab9862fef7"
server_key_version: "encryptionkey@0"
I20260808 22:26:55.758519 11935 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:26:55.769835 11935 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:26:55.773164 11935 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:26:55.773370 11935 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:26:55.773931 11935 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:26:55.774741 11935 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:26:55.774816 11935 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:55.774886 11935 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:26:55.774936 11935 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:55.783825 11935 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.194:36065
I20260808 22:26:55.783864 12156 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.194:36065 every 8 connection(s)
I20260808 22:26:55.784687 11935 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
I20260808 22:26:55.787671 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 11935
I20260808 22:26:55.787815 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance
I20260808 22:26:55.788108 5167 external_mini_cluster.cc:1428] Setting key a179a8e382c601c0ed0dce98cc902e3c
Aug 08 22:26:55 dist-test-slave-7f01 krb5kdc[11508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228015, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.11.195@KRBTEST.COM; defaulting to no policy
I20260808 22:26:55.798920 11644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:55.786440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:36763 (local address 127.5.11.254:41873)
0808 22:26:55.786571 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:55.786575 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:55.787323 (+ 748us) server_negotiation.cc:407] Connection header received
0808 22:26:55.789031 (+ 1708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:55.789034 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:55.789080 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:55.789153 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:55.790471 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:55.790992 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:55.792217 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:55.792441 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:55.792557 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:55.795278 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:55.795281 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:55.795283 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:55.795307 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:55.796781 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:55.797304 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:55.797308 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:55.797310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:55.797396 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:55.797752 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:55.797756 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:55.797758 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:55.797946 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:55.798063 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:55.798622 (+ 559us) server_negotiation.cc:299] Negotiation successful
0808 22:26:55.798747 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 22:26:55.799790 12161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:55.786739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:41873 (local address 127.5.11.194:36763)
0808 22:26:55.787176 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:55.787218 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:55.788791 (+ 1573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:55.789346 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:55.789357 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:55.789852 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:55.790322 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:55.790335 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:55.791129 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:55.791132 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:55.792090 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:55.792100 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:55.792607 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:55.792611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:55.792774 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:55.793371 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:26:55.793388 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:55.795107 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:55.796907 (+ 1800us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:55.796914 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:55.796928 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:55.797196 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:55.797494 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:55.797498 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:55.797500 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:55.797649 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:55.798073 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:55.798082 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:55.798364 (+ 282us) client_negotiation.cc:769] Sending connection context
0808 22:26:55.798560 (+ 196us) client_negotiation.cc:240] Negotiation successful
0808 22:26:55.798872 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":106,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.195@KRBTEST.COM" created.
I20260808 22:26:55.801103 12157 heartbeater.cc:347] Connected to a master server at 127.5.11.254:41873
I20260808 22:26:55.801376 12157 heartbeater.cc:464] Registering TS with master...
I20260808 22:26:55.801981 12157 heartbeater.cc:511] Master 127.5.11.254:41873 requested a full tablet report, sending...
I20260808 22:26:55.803130 11585 ts_manager.cc:194] Registered new tserver with Master: f6a89330010b48ae8bd11523b4a9779a (127.5.11.194:36065)
I20260808 22:26:55.803728 11585 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.194@KRBTEST.COM'} at 127.5.11.194:36763
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab.
Entry for principal kudu/127.5.11.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab.
I20260808 22:26:55.815618 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.195:0
--local_ip_for_outbound_sockets=127.5.11.195
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:41873
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:34309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:26:55.930352 12166 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:26:55.930598 12166 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:26:55.930640 12166 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:26:55.934826 12166 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:26:55.934892 12166 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:26:55.934998 12166 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.195
I20260808 22:26:55.940276 12166 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:34309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:41873
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12166
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.195
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:26:55.941267 12166 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:26:55.942250 12166 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:26:55.951712 12174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:55.951793 12173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:55.951787 12166 server_base.cc:1034] running on GCE node
W20260808 22:26:55.951781 12179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:55.952390 12166 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:26:55.953006 12166 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:26:55.954205 12166 hybrid_clock.cc:648] HybridClock initialized: now 1786228015954183 us; error 32 us; skew 500 ppm
Aug 08 22:26:55 dist-test-slave-7f01 krb5kdc[11508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228015, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:26:55.957288 12166 init.cc:377] Logged in from keytab as kudu/127.5.11.195@KRBTEST.COM (short username kudu)
I20260808 22:26:55.958179 12166 webserver.cc:460] Webserver started at http://127.5.11.195:44333/ using document root <none> and password file <none>
I20260808 22:26:55.958746 12166 fs_manager.cc:359] Metadata directory not provided
I20260808 22:26:55.958840 12166 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:26:55.959059 12166 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:26:55.961293 12166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-2/data/instance:
uuid: "a856ca4d5ee54ab2a31b67076e1e9656"
format_stamp: "Formatted at 2026-08-08 22:26:55 on dist-test-slave-7f01"
server_key: "e89136cc5c39d07f6d8d82a194b80969"
server_key_iv: "f60bf2b967525771a3c8e7187c2ef6d5"
server_key_version: "encryptionkey@0"
I20260808 22:26:55.961928 12166 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance:
uuid: "a856ca4d5ee54ab2a31b67076e1e9656"
format_stamp: "Formatted at 2026-08-08 22:26:55 on dist-test-slave-7f01"
server_key: "e89136cc5c39d07f6d8d82a194b80969"
server_key_iv: "f60bf2b967525771a3c8e7187c2ef6d5"
server_key_version: "encryptionkey@0"
I20260808 22:26:55.965808 12166 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260808 22:26:55.968374 12189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:55.969619 12166 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 22:26:55.969761 12166 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-2/wal
uuid: "a856ca4d5ee54ab2a31b67076e1e9656"
format_stamp: "Formatted at 2026-08-08 22:26:55 on dist-test-slave-7f01"
server_key: "e89136cc5c39d07f6d8d82a194b80969"
server_key_iv: "f60bf2b967525771a3c8e7187c2ef6d5"
server_key_version: "encryptionkey@0"
I20260808 22:26:55.969874 12166 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:26:55.990761 12166 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:26:55.994323 12166 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:26:55.994539 12166 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:26:55.995138 12166 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:26:55.996052 12166 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:26:55.996124 12166 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:55.996196 12166 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:26:55.996248 12166 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:56.005303 12166 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.195:35335
I20260808 22:26:56.005334 12384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.195:35335 every 8 connection(s)
I20260808 22:26:56.006192 12166 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
I20260808 22:26:56.012120 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 12166
I20260808 22:26:56.012274 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance
I20260808 22:26:56.012562 5167 external_mini_cluster.cc:1428] Setting key c2bb1ce67613fa5547a7a88bbe922343
Aug 08 22:26:56 dist-test-slave-7f01 krb5kdc[11508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228015, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:26:56.020334 11644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:56.008004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:49955 (local address 127.5.11.254:41873)
0808 22:26:56.008162 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:56.008167 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:56.009179 (+ 1012us) server_negotiation.cc:407] Connection header received
0808 22:26:56.010658 (+ 1479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:56.010660 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:56.010709 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:56.010785 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:56.012016 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:56.012658 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:56.014145 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:56.014370 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:56.014486 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:56.016970 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:56.016973 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:56.016975 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:56.017000 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:56.018522 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:56.018957 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:56.018960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:56.018961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:56.019041 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:56.019333 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:56.019335 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:56.019336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:56.019504 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:56.019597 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:56.020110 (+ 513us) server_negotiation.cc:299] Negotiation successful
0808 22:26:56.020214 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 22:26:56.021098 12388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:56.008499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:41873 (local address 127.5.11.195:49955)
0808 22:26:56.008987 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:56.009046 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:56.010453 (+ 1407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:56.010956 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:56.010962 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:56.011306 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:56.011857 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:56.011873 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:56.012794 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:56.012797 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:56.014023 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:56.014033 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:56.014531 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:56.014535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:56.014716 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:56.015284 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:26:56.015301 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:56.016815 (+ 1514us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:56.018619 (+ 1804us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:56.018623 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:56.018633 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:56.018873 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:56.019121 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:56.019124 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:56.019125 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:56.019253 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:56.019606 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:56.019611 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:56.019891 (+ 280us) client_negotiation.cc:769] Sending connection context
0808 22:26:56.020055 (+ 164us) client_negotiation.cc:240] Negotiation successful
0808 22:26:56.020291 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":113,"threads_started":1}
I20260808 22:26:56.022231 12386 heartbeater.cc:347] Connected to a master server at 127.5.11.254:41873
I20260808 22:26:56.022467 12386 heartbeater.cc:464] Registering TS with master...
I20260808 22:26:56.022948 12386 heartbeater.cc:511] Master 127.5.11.254:41873 requested a full tablet report, sending...
I20260808 22:26:56.024039 11585 ts_manager.cc:194] Registered new tserver with Master: a856ca4d5ee54ab2a31b67076e1e9656 (127.5.11.195:35335)
I20260808 22:26:56.024555 11585 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.195@KRBTEST.COM'} at 127.5.11.195:49955
I20260808 22:26:56.026494 5167 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 08 22:26:56 dist-test-slave-7f01 krb5kdc[11508](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 22:26:56 dist-test-slave-7f01 krb5kdc[11508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.193@KRBTEST.COM
I20260808 22:26:56.054096 12397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:56.044701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54102 (local address 127.5.11.193:44857)
0808 22:26:56.045002 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:56.045018 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:56.045046 (+ 28us) server_negotiation.cc:407] Connection header received
0808 22:26:56.045113 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:56.045119 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:56.045291 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:56.045447 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:56.046167 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:56.046943 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:56.047881 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:56.048249 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:56.048374 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:56.050665 (+ 2291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:56.050674 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:56.050686 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:56.050711 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:56.052211 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:56.052666 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:56.052672 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:56.052673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:56.052755 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:56.053044 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:56.053048 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:56.053050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:56.053290 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:56.053530 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:56.053763 (+ 233us) server_negotiation.cc:299] Negotiation successful
0808 22:26:56.053918 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":86,"threads_started":1}
Aug 08 22:26:56 dist-test-slave-7f01 krb5kdc[11508](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228016, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:26:56.064771 11644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:56.056954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43076 (local address 127.5.11.254:41873)
0808 22:26:56.057102 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:56.057111 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:56.057125 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:26:56.057225 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:56.057227 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:56.057270 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:56.057368 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:56.057928 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:56.058566 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:56.059298 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:56.059515 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:56.059593 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:56.061945 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:56.061948 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:56.061950 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:56.061973 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:56.063232 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:56.063600 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:56.063603 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:56.063604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:56.063676 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:56.063966 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:56.063968 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:56.063969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:56.064186 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:56.064289 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:56.064474 (+ 185us) server_negotiation.cc:299] Negotiation successful
0808 22:26:56.064589 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 22:26:56.065212 11585 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43076
I20260808 22:26:56.065847 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 11687
I20260808 22:26:56.082695 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 11935
I20260808 22:26:56.094485 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 12166
I20260808 22:26:56.109501 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 11526
2026-08-08T22:26:56Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2528 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:26:56 dist-test-slave-7f01 krb5kdc[12407](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:26:56 dist-test-slave-7f01 krb5kdc[12407](info): set up 2 sockets
Aug 08 22:26:56 dist-test-slave-7f01 krb5kdc[12407](info): commencing operation
krb5kdc: starting...
W20260808 22:26:57.370209 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.232s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:26:57 dist-test-slave-7f01 krb5kdc[12407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:26:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:26:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.11.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
I20260808 22:26:57.495535 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:37325
--webserver_interface=127.5.11.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:39873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:37325
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:26:57.611653 12432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:26:57.611986 12432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:26:57.612082 12432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:26:57.616129 12432 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:26:57.616223 12432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:26:57.616262 12432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:26:57.616300 12432 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:26:57.616340 12432 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:26:57.621583 12432 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:39873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:37325
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:37325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:26:57.622663 12432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:26:57.623713 12432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:26:57.632192 12444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:57.632174 12448 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:57.632329 12432 server_base.cc:1034] running on GCE node
W20260808 22:26:57.632174 12445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:57.633107 12432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:26:57.634147 12432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:26:57.635339 12432 hybrid_clock.cc:648] HybridClock initialized: now 1786228017635311 us; error 31 us; skew 500 ppm
Aug 08 22:26:57 dist-test-slave-7f01 krb5kdc[12407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228017, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:26:57.638154 12432 init.cc:377] Logged in from keytab as kudu/127.5.11.254@KRBTEST.COM (short username kudu)
I20260808 22:26:57.638892 12432 fs_manager.cc:359] Metadata directory not provided
I20260808 22:26:57.638958 12432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:26:57.639132 12432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:26:57.641217 12432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "1867f778f7d04a969d19293a38bebe6d"
format_stamp: "Formatted at 2026-08-08 22:26:57 on dist-test-slave-7f01"
server_key: "671382a970f52a5748b36165389830e3"
server_key_iv: "63895c999db8353a8ad1700b8909ac1d"
server_key_version: "encryptionkey@0"
I20260808 22:26:57.641800 12432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "1867f778f7d04a969d19293a38bebe6d"
format_stamp: "Formatted at 2026-08-08 22:26:57 on dist-test-slave-7f01"
server_key: "671382a970f52a5748b36165389830e3"
server_key_iv: "63895c999db8353a8ad1700b8909ac1d"
server_key_version: "encryptionkey@0"
I20260808 22:26:57.645676 12432 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 22:26:57.648267 12463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:57.649480 12432 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:26:57.649643 12432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "1867f778f7d04a969d19293a38bebe6d"
format_stamp: "Formatted at 2026-08-08 22:26:57 on dist-test-slave-7f01"
server_key: "671382a970f52a5748b36165389830e3"
server_key_iv: "63895c999db8353a8ad1700b8909ac1d"
server_key_version: "encryptionkey@0"
I20260808 22:26:57.649755 12432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:26:57.666883 12432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:26:57.669749 12432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:26:57.669951 12432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:26:57.676999 12432 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:37325
I20260808 22:26:57.677266 12544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:37325 every 8 connection(s)
I20260808 22:26:57.677667 12432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:26:57.680462 12545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:57.682261 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 12432
I20260808 22:26:57.682401 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:26:57.682672 5167 external_mini_cluster.cc:1428] Setting key 4d39a8835adf007d62994b4f12b21ac9
I20260808 22:26:57.686674 12545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1867f778f7d04a969d19293a38bebe6d: Bootstrap starting.
I20260808 22:26:57.689023 12545 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1867f778f7d04a969d19293a38bebe6d: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:57.689854 12545 log.cc:824] T 00000000000000000000000000000000 P 1867f778f7d04a969d19293a38bebe6d: Log is configured to *not* fsync() on all Append() calls
Aug 08 22:26:57 dist-test-slave-7f01 krb5kdc[12407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228017, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:26:57.691905 12545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1867f778f7d04a969d19293a38bebe6d: No bootstrap required, opened a new log
I20260808 22:26:57.695663 12545 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1867f778f7d04a969d19293a38bebe6d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1867f778f7d04a969d19293a38bebe6d" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 37325 } }
I20260808 22:26:57.695963 12545 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1867f778f7d04a969d19293a38bebe6d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:57.696066 12545 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1867f778f7d04a969d19293a38bebe6d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1867f778f7d04a969d19293a38bebe6d, State: Initialized, Role: FOLLOWER
I20260808 22:26:57.696496 12545 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1867f778f7d04a969d19293a38bebe6d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1867f778f7d04a969d19293a38bebe6d" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 37325 } }
I20260808 22:26:57.696557 12548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:57.684041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39434 (local address 127.5.11.254:37325)
0808 22:26:57.684490 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:57.684508 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:57.684541 (+ 33us) server_negotiation.cc:407] Connection header received
0808 22:26:57.685180 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:57.685196 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:57.685510 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:57.685844 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:57.686666 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:57.687700 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:57.688614 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:57.689141 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:57.689315 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:57.691848 (+ 2533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:57.691853 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:57.691866 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:57.691892 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:57.693568 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:57.694067 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:57.694073 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:57.694077 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:57.694187 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:57.694476 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:57.694480 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:57.694482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:57.694859 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:57.695092 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:57.695339 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 22:26:57.695619 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":134,"threads_started":1}
I20260808 22:26:57.696668 12545 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1867f778f7d04a969d19293a38bebe6d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:26:57.696743 12545 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1867f778f7d04a969d19293a38bebe6d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:26:57.696871 12545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1867f778f7d04a969d19293a38bebe6d [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:57.698030 12545 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1867f778f7d04a969d19293a38bebe6d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1867f778f7d04a969d19293a38bebe6d" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 37325 } }
I20260808 22:26:57.698388 12545 leader_election.cc:302] T 00000000000000000000000000000000 P 1867f778f7d04a969d19293a38bebe6d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1867f778f7d04a969d19293a38bebe6d; no voters:
I20260808 22:26:57.698747 12545 leader_election.cc:288] T 00000000000000000000000000000000 P 1867f778f7d04a969d19293a38bebe6d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:26:57.698835 12551 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1867f778f7d04a969d19293a38bebe6d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:26:57.699020 12551 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1867f778f7d04a969d19293a38bebe6d [term 1 LEADER]: Becoming Leader. State: Replica: 1867f778f7d04a969d19293a38bebe6d, State: Running, Role: LEADER
I20260808 22:26:57.699371 12551 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1867f778f7d04a969d19293a38bebe6d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1867f778f7d04a969d19293a38bebe6d" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 37325 } }
I20260808 22:26:57.700033 12545 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1867f778f7d04a969d19293a38bebe6d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:26:57.700593 12554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1867f778f7d04a969d19293a38bebe6d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1867f778f7d04a969d19293a38bebe6d. Latest consensus state: current_term: 1 leader_uuid: "1867f778f7d04a969d19293a38bebe6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1867f778f7d04a969d19293a38bebe6d" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 37325 } } }
I20260808 22:26:57.700814 12554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1867f778f7d04a969d19293a38bebe6d [sys.catalog]: This master's current role is: LEADER
I20260808 22:26:57.701139 12553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1867f778f7d04a969d19293a38bebe6d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1867f778f7d04a969d19293a38bebe6d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1867f778f7d04a969d19293a38bebe6d" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 37325 } } }
I20260808 22:26:57.701243 12553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1867f778f7d04a969d19293a38bebe6d [sys.catalog]: This master's current role is: LEADER
I20260808 22:26:57.702642 12560 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:26:57.705432 12560 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:26:57.710417 12560 catalog_manager.cc:1347] Generated new cluster ID: 343689dc4c7344ebac249f0acb577bfb
I20260808 22:26:57.710482 12560 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:26:57.747790 12560 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:26:57.749141 12560 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:26:57.756222 12560 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1867f778f7d04a969d19293a38bebe6d: Generated new TSK 0
I20260808 22:26:57.757018 12560 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.11.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab.
Entry for principal kudu/127.5.11.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab.
I20260808 22:26:57.797189 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.193:0
--local_ip_for_outbound_sockets=127.5.11.193
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:37325
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:39873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:26:57.914386 12591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:26:57.914639 12591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:26:57.914683 12591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:26:57.918610 12591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:26:57.918666 12591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:26:57.918725 12591 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.193
I20260808 22:26:57.923888 12591 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:39873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:37325
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12591
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.193
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:26:57.924937 12591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:26:57.925953 12591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:26:57.935163 12602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:57.935158 12598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:57.935185 12599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:57.935636 12591 server_base.cc:1034] running on GCE node
I20260808 22:26:57.936159 12591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:26:57.936920 12591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:26:57.938143 12591 hybrid_clock.cc:648] HybridClock initialized: now 1786228017938109 us; error 47 us; skew 500 ppm
Aug 08 22:26:57 dist-test-slave-7f01 krb5kdc[12407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228017, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:26:57.941264 12591 init.cc:377] Logged in from keytab as kudu/127.5.11.193@KRBTEST.COM (short username kudu)
I20260808 22:26:57.942034 12591 fs_manager.cc:359] Metadata directory not provided
I20260808 22:26:57.942130 12591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:26:57.942353 12591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:26:57.944813 12591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-0/data/instance:
uuid: "bc98f64fcfed4433b0b157fff0f75ffe"
format_stamp: "Formatted at 2026-08-08 22:26:57 on dist-test-slave-7f01"
server_key: "2d34c76f9a9f82f9a8196c0255cba2e1"
server_key_iv: "6176449832ba274af346a1d9ec4dcb7c"
server_key_version: "encryptionkey@0"
I20260808 22:26:57.945508 12591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance:
uuid: "bc98f64fcfed4433b0b157fff0f75ffe"
format_stamp: "Formatted at 2026-08-08 22:26:57 on dist-test-slave-7f01"
server_key: "2d34c76f9a9f82f9a8196c0255cba2e1"
server_key_iv: "6176449832ba274af346a1d9ec4dcb7c"
server_key_version: "encryptionkey@0"
I20260808 22:26:57.949110 12591 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:26:57.951361 12611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:57.952628 12591 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:26:57.952781 12591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-0/wal
uuid: "bc98f64fcfed4433b0b157fff0f75ffe"
format_stamp: "Formatted at 2026-08-08 22:26:57 on dist-test-slave-7f01"
server_key: "2d34c76f9a9f82f9a8196c0255cba2e1"
server_key_iv: "6176449832ba274af346a1d9ec4dcb7c"
server_key_version: "encryptionkey@0"
I20260808 22:26:57.952876 12591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:26:57.973554 12591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:26:57.976488 12591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:26:57.976640 12591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:26:57.977154 12591 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:26:57.977941 12591 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:26:57.977990 12591 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:57.978030 12591 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:26:57.978046 12591 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:57.987097 12591 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.193:36091
I20260808 22:26:57.987124 12794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.193:36091 every 8 connection(s)
I20260808 22:26:57.987685 12591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
I20260808 22:26:57.994501 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 12591
I20260808 22:26:57.994628 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance
I20260808 22:26:57.994903 5167 external_mini_cluster.cc:1428] Setting key 071eed45b0b5a8d3823346287fe188cb
Aug 08 22:26:57 dist-test-slave-7f01 krb5kdc[12407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228017, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:26:58.001844 12548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:57.989442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:33069 (local address 127.5.11.254:37325)
0808 22:26:57.989590 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:57.989594 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:57.990698 (+ 1104us) server_negotiation.cc:407] Connection header received
0808 22:26:57.991603 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:57.991606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:57.991668 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:57.991810 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:57.992985 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:57.993524 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:57.994675 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:57.994913 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:57.995030 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:57.997581 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:57.997584 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:57.997587 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:57.997616 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:57.999300 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:57.999914 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:57.999921 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:57.999924 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:58.000026 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:58.000422 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:58.000429 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:58.000433 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:58.000707 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:58.000816 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:58.001370 (+ 554us) server_negotiation.cc:299] Negotiation successful
0808 22:26:58.001585 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 22:26:58.002691 12798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:57.989731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:37325 (local address 127.5.11.193:33069)
0808 22:26:57.990532 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:57.990573 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:57.991390 (+ 817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:57.992009 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:57.992016 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:57.992368 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:57.992836 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:57.992850 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:57.993655 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:57.993658 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:57.994518 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:57.994529 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:57.995077 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:57.995081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:57.995271 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:57.995934 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:26:57.995952 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:57.997412 (+ 1460us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:57.999443 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:57.999448 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:57.999460 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:57.999804 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:58.000176 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:58.000179 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:58.000180 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:58.000321 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:58.000817 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:58.000822 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:58.001073 (+ 251us) client_negotiation.cc:769] Sending connection context
0808 22:26:58.001309 (+ 236us) client_negotiation.cc:240] Negotiation successful
0808 22:26:58.001658 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":122,"threads_started":1}
I20260808 22:26:58.004102 12796 heartbeater.cc:347] Connected to a master server at 127.5.11.254:37325
I20260808 22:26:58.004376 12796 heartbeater.cc:464] Registering TS with master...
I20260808 22:26:58.004947 12796 heartbeater.cc:511] Master 127.5.11.254:37325 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.5.11.194@KRBTEST.COM; defaulting to no policy
I20260808 22:26:58.006672 12489 ts_manager.cc:194] Registered new tserver with Master: bc98f64fcfed4433b0b157fff0f75ffe (127.5.11.193:36091)
I20260808 22:26:58.007953 12489 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.193@KRBTEST.COM'} at 127.5.11.193:33069
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab.
Entry for principal kudu/127.5.11.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab.
I20260808 22:26:58.027294 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.194:0
--local_ip_for_outbound_sockets=127.5.11.194
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:37325
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:39873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:26:58.140537 12803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:26:58.140766 12803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:26:58.140812 12803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:26:58.144716 12803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:26:58.144778 12803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:26:58.144876 12803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.194
I20260808 22:26:58.149948 12803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:39873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:37325
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12803
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.194
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:26:58.150939 12803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:26:58.152025 12803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:26:58.161662 12813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:58.161804 12809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:58.161656 12810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:58.162030 12803 server_base.cc:1034] running on GCE node
I20260808 22:26:58.162418 12803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:26:58.163019 12803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:26:58.164191 12803 hybrid_clock.cc:648] HybridClock initialized: now 1786228018164174 us; error 31 us; skew 500 ppm
Aug 08 22:26:58 dist-test-slave-7f01 krb5kdc[12407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228018, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:26:58.166962 12803 init.cc:377] Logged in from keytab as kudu/127.5.11.194@KRBTEST.COM (short username kudu)
I20260808 22:26:58.167727 12803 fs_manager.cc:359] Metadata directory not provided
I20260808 22:26:58.167881 12803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:26:58.168120 12803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:26:58.170081 12803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-1/data/instance:
uuid: "acdb40acba534463877dc53f2dddad1c"
format_stamp: "Formatted at 2026-08-08 22:26:58 on dist-test-slave-7f01"
server_key: "f1e17906cf189beac879115a12de1dcd"
server_key_iv: "e7e6188fda639637b766c7ff1a14da77"
server_key_version: "encryptionkey@0"
I20260808 22:26:58.170699 12803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance:
uuid: "acdb40acba534463877dc53f2dddad1c"
format_stamp: "Formatted at 2026-08-08 22:26:58 on dist-test-slave-7f01"
server_key: "f1e17906cf189beac879115a12de1dcd"
server_key_iv: "e7e6188fda639637b766c7ff1a14da77"
server_key_version: "encryptionkey@0"
I20260808 22:26:58.174503 12803 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260808 22:26:58.176919 12823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:58.178100 12803 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:26:58.178243 12803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-1/wal
uuid: "acdb40acba534463877dc53f2dddad1c"
format_stamp: "Formatted at 2026-08-08 22:26:58 on dist-test-slave-7f01"
server_key: "f1e17906cf189beac879115a12de1dcd"
server_key_iv: "e7e6188fda639637b766c7ff1a14da77"
server_key_version: "encryptionkey@0"
I20260808 22:26:58.178360 12803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:26:58.189005 12803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:26:58.191725 12803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:26:58.192008 12803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:26:58.192554 12803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:26:58.193352 12803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:26:58.193425 12803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:58.193492 12803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:26:58.193547 12803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:58.202801 12803 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.194:36643
I20260808 22:26:58.202828 13020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.194:36643 every 8 connection(s)
I20260808 22:26:58.203344 12803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
I20260808 22:26:58.204631 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 12803
I20260808 22:26:58.204749 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance
I20260808 22:26:58.205112 5167 external_mini_cluster.cc:1428] Setting key dbcb532ce532b1c0e2533b7038f437e7
Aug 08 22:26:58 dist-test-slave-7f01 krb5kdc[12407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228018, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.11.195@KRBTEST.COM; defaulting to no policy
I20260808 22:26:58.218032 12548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:58.205122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:43321 (local address 127.5.11.254:37325)
0808 22:26:58.205257 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:58.205261 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:58.206764 (+ 1503us) server_negotiation.cc:407] Connection header received
0808 22:26:58.207693 (+ 929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:58.207699 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:58.207800 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:58.207887 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:58.209122 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:58.209637 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:58.211106 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:58.211531 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:58.211663 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:58.214135 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:58.214142 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:58.214148 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:58.214184 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:58.215936 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:58.216430 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:58.216436 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:58.216440 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:58.216549 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:58.216895 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:58.216902 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:58.216907 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:58.217199 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:58.217318 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:58.217771 (+ 453us) server_negotiation.cc:299] Negotiation successful
0808 22:26:58.217883 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 22:26:58.218820 13023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:58.205592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:37325 (local address 127.5.11.194:43321)
0808 22:26:58.206592 (+ 1000us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:58.206649 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:58.207453 (+ 804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:58.208034 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:58.208040 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:58.208376 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:58.208956 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:58.208979 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:58.209765 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:58.209772 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:58.210953 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:58.210964 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:58.211683 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:58.211686 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:58.211900 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:58.212529 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:26:58.212545 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:58.213956 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:58.216049 (+ 2093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:58.216054 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:58.216065 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:58.216315 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:58.216653 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:58.216655 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:58.216656 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:58.216795 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:58.217308 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:58.217313 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:58.217548 (+ 235us) client_negotiation.cc:769] Sending connection context
0808 22:26:58.217706 (+ 158us) client_negotiation.cc:240] Negotiation successful
0808 22:26:58.217946 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":149,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.195@KRBTEST.COM" created.
I20260808 22:26:58.220085 13021 heartbeater.cc:347] Connected to a master server at 127.5.11.254:37325
I20260808 22:26:58.220341 13021 heartbeater.cc:464] Registering TS with master...
I20260808 22:26:58.220909 13021 heartbeater.cc:511] Master 127.5.11.254:37325 requested a full tablet report, sending...
I20260808 22:26:58.222118 12489 ts_manager.cc:194] Registered new tserver with Master: acdb40acba534463877dc53f2dddad1c (127.5.11.194:36643)
I20260808 22:26:58.222725 12489 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.194@KRBTEST.COM'} at 127.5.11.194:43321
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab.
Entry for principal kudu/127.5.11.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab.
I20260808 22:26:58.236249 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.195:0
--local_ip_for_outbound_sockets=127.5.11.195
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:37325
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:39873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:26:58.348439 13033 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:26:58.348716 13033 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:26:58.348793 13033 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:26:58.352766 13033 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:26:58.352854 13033 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:26:58.352939 13033 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.195
I20260808 22:26:58.358310 13033 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:39873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:37325
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13033
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.195
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:26:58.359304 13033 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:26:58.360303 13033 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:26:58.369493 13040 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:58.369501 13044 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:58.369500 13042 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:58.369725 13033 server_base.cc:1034] running on GCE node
I20260808 22:26:58.370203 13033 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:26:58.370831 13033 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:26:58.372038 13033 hybrid_clock.cc:648] HybridClock initialized: now 1786228018372007 us; error 28 us; skew 500 ppm
Aug 08 22:26:58 dist-test-slave-7f01 krb5kdc[12407](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228018, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:26:58.374900 13033 init.cc:377] Logged in from keytab as kudu/127.5.11.195@KRBTEST.COM (short username kudu)
I20260808 22:26:58.375576 13033 fs_manager.cc:359] Metadata directory not provided
I20260808 22:26:58.375641 13033 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:26:58.375878 13033 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:26:58.377811 13033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-2/data/instance:
uuid: "7517d4c940054cfb948102ef80da83e4"
format_stamp: "Formatted at 2026-08-08 22:26:58 on dist-test-slave-7f01"
server_key: "a4438deee8a437e66cb1d3b79f501d0b"
server_key_iv: "b9a517e233551fa3047b57298a2c67b1"
server_key_version: "encryptionkey@0"
I20260808 22:26:58.378329 13033 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance:
uuid: "7517d4c940054cfb948102ef80da83e4"
format_stamp: "Formatted at 2026-08-08 22:26:58 on dist-test-slave-7f01"
server_key: "a4438deee8a437e66cb1d3b79f501d0b"
server_key_iv: "b9a517e233551fa3047b57298a2c67b1"
server_key_version: "encryptionkey@0"
I20260808 22:26:58.382114 13033 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 22:26:58.384583 13053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:58.386510 13033 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:26:58.386646 13033 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-2/wal
uuid: "7517d4c940054cfb948102ef80da83e4"
format_stamp: "Formatted at 2026-08-08 22:26:58 on dist-test-slave-7f01"
server_key: "a4438deee8a437e66cb1d3b79f501d0b"
server_key_iv: "b9a517e233551fa3047b57298a2c67b1"
server_key_version: "encryptionkey@0"
I20260808 22:26:58.386792 13033 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:26:58.403133 13033 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:26:58.405864 13033 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:26:58.406064 13033 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:26:58.406610 13033 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:26:58.407445 13033 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:26:58.407517 13033 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:58.407588 13033 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:26:58.407639 13033 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:58.416280 13033 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.195:44099
I20260808 22:26:58.416304 13245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.195:44099 every 8 connection(s)
I20260808 22:26:58.416854 13033 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
I20260808 22:26:58.422901 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 13033
I20260808 22:26:58.423060 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance
I20260808 22:26:58.423388 5167 external_mini_cluster.cc:1428] Setting key 8e69a7c4c28e1dcc469bf99db57a3721
Aug 08 22:26:58 dist-test-slave-7f01 krb5kdc[12407](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228018, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:26:58.430253 12548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:58.418610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:33871 (local address 127.5.11.254:37325)
0808 22:26:58.418751 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:58.418755 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:58.419966 (+ 1211us) server_negotiation.cc:407] Connection header received
0808 22:26:58.420788 (+ 822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:58.420790 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:58.420838 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:58.420912 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:58.422017 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:58.422533 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:58.424164 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:58.424468 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:58.424586 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:58.427038 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:58.427041 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:58.427043 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:58.427066 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:58.428357 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:58.428812 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:58.428815 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:58.428816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:58.428891 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:58.429272 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:58.429275 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:58.429278 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:58.429489 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:58.429570 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:58.430042 (+ 472us) server_negotiation.cc:299] Negotiation successful
0808 22:26:58.430137 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 22:26:58.431083 13254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:58.418978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:37325 (local address 127.5.11.195:33871)
0808 22:26:58.419825 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:58.419867 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:58.420608 (+ 741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:58.421076 (+ 468us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:58.421082 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:58.421419 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:58.421888 (+ 469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:58.421901 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:58.422670 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:58.422675 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:58.424055 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:58.424067 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:58.424636 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:58.424640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:58.424810 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:58.425405 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:26:58.425427 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:58.426883 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:58.428461 (+ 1578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:58.428466 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:58.428477 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:58.428724 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:58.429000 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:26:58.429004 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:26:58.429005 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:26:58.429169 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:26:58.429586 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:58.429591 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:58.429826 (+ 235us) client_negotiation.cc:769] Sending connection context
0808 22:26:58.429989 (+ 163us) client_negotiation.cc:240] Negotiation successful
0808 22:26:58.430244 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":114,"threads_started":1}
I20260808 22:26:58.432205 13249 heartbeater.cc:347] Connected to a master server at 127.5.11.254:37325
I20260808 22:26:58.432443 13249 heartbeater.cc:464] Registering TS with master...
I20260808 22:26:58.432885 13249 heartbeater.cc:511] Master 127.5.11.254:37325 requested a full tablet report, sending...
I20260808 22:26:58.433902 12489 ts_manager.cc:194] Registered new tserver with Master: 7517d4c940054cfb948102ef80da83e4 (127.5.11.195:44099)
I20260808 22:26:58.434482 12489 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.195@KRBTEST.COM'} at 127.5.11.195:33871
I20260808 22:26:58.437206 5167 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:26:58.446559 12548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:58.439465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39438 (local address 127.5.11.254:37325)
0808 22:26:58.439588 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:58.439591 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:58.439861 (+ 270us) server_negotiation.cc:407] Connection header received
0808 22:26:58.439970 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:58.439972 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:58.440010 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:58.440073 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:58.440769 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:58.441242 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:58.442098 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:58.442295 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:58.442367 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:58.443643 (+ 1276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:58.443646 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:58.443648 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:26:58.443669 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:58.445129 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:58.445533 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:58.445536 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:58.445537 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:58.445609 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:26:58.445857 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:26:58.445859 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:26:58.445860 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:26:58.446055 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:58.446161 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:58.446357 (+ 196us) server_negotiation.cc:299] Negotiation successful
0808 22:26:58.446446 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 22:26:58.449266 12489 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39438:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:26:58.452441 12489 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:26:58.467360 13269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:58.461354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37728 (local address 127.5.11.195:44099)
0808 22:26:58.461889 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:58.462336 (+ 447us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:58.462366 (+ 30us) server_negotiation.cc:407] Connection header received
0808 22:26:58.462561 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:58.462566 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:58.462786 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:58.462944 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:26:58.463235 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:58.464220 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:58.465949 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:58.466631 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:58.466763 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:58.466935 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:58.467104 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 22:26:58.467197 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":187,"threads_started":1}
I20260808 22:26:58.467355 13266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:58.461215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:44099 (local address 127.0.0.1:37728)
0808 22:26:58.462204 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:58.462257 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:58.462391 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:58.462945 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:58.462951 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:58.462976 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:26:58.463108 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:58.463115 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:58.464352 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:58.464356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:58.465807 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:58.465819 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:58.466780 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:58.466784 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:58.466893 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:58.466929 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 22:26:58.467072 (+ 143us) client_negotiation.cc:240] Negotiation successful
0808 22:26:58.467196 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"mutex_wait_us":4,"thread_start_us":99,"threads_started":1}
I20260808 22:26:58.468276 13273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:58.462406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55288 (local address 127.5.11.194:36643)
0808 22:26:58.462829 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:58.462841 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:58.462865 (+ 24us) server_negotiation.cc:407] Connection header received
0808 22:26:58.462945 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:58.462950 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:58.463157 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:58.463390 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:26:58.463628 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:58.464832 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:58.466344 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:58.467111 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:58.467839 (+ 728us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:58.467989 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:58.468039 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 22:26:58.468127 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":107,"threads_started":1}
I20260808 22:26:58.467657 13267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:58.461225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:36643 (local address 127.0.0.1:55288)
0808 22:26:58.462572 (+ 1347us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:58.462585 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:58.462672 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:58.463370 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:58.463373 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:58.463393 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:26:58.463498 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:58.463516 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:58.464998 (+ 1482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:58.465004 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:58.466207 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:58.466219 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:58.467243 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:58.467247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:58.467343 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:58.467371 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 22:26:58.467424 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 22:26:58.467520 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1285,"mutex_wait_us":62,"thread_start_us":77,"threads_started":1}
I20260808 22:26:58.468592 13271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:58.461978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:36091 (local address 127.0.0.1:53388)
0808 22:26:58.462654 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:58.462669 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:58.462757 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:58.463597 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:58.463600 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:58.463618 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:26:58.463703 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:58.463708 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:58.465075 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:58.465078 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:58.466093 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:58.466105 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:58.467414 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:58.467757 (+ 343us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:58.467851 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:58.467881 (+ 30us) client_negotiation.cc:769] Sending connection context
0808 22:26:58.467954 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 22:26:58.468039 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"mutex_wait_us":63,"thread_start_us":113,"threads_started":1}
I20260808 22:26:58.468724 13272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:58.462166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53388 (local address 127.5.11.193:36091)
0808 22:26:58.462925 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:58.462932 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:58.462953 (+ 21us) server_negotiation.cc:407] Connection header received
0808 22:26:58.463040 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:58.463045 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:58.463274 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:58.463427 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:26:58.463843 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:58.464935 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:58.466228 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:58.467080 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:58.467204 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:58.467351 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:58.468119 (+ 768us) server_negotiation.cc:299] Negotiation successful
0808 22:26:58.468231 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":109,"threads_started":1}
I20260808 22:26:58.470539 13138 tablet_service.cc:1467] Processing CreateTablet for tablet 02a794c13a1741688e1ffc22ba22dba1 (DEFAULT_TABLE table=test-table [id=93071044b64d45cbbd102ebab1a4bc21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:26:58.471179 12686 tablet_service.cc:1467] Processing CreateTablet for tablet 02a794c13a1741688e1ffc22ba22dba1 (DEFAULT_TABLE table=test-table [id=93071044b64d45cbbd102ebab1a4bc21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:26:58.471761 13138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02a794c13a1741688e1ffc22ba22dba1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:58.471685 12905 tablet_service.cc:1467] Processing CreateTablet for tablet 02a794c13a1741688e1ffc22ba22dba1 (DEFAULT_TABLE table=test-table [id=93071044b64d45cbbd102ebab1a4bc21]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:26:58.472221 12686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02a794c13a1741688e1ffc22ba22dba1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:58.472683 12905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02a794c13a1741688e1ffc22ba22dba1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:58.478466 13274 tablet_bootstrap.cc:492] T 02a794c13a1741688e1ffc22ba22dba1 P bc98f64fcfed4433b0b157fff0f75ffe: Bootstrap starting.
I20260808 22:26:58.479430 13276 tablet_bootstrap.cc:492] T 02a794c13a1741688e1ffc22ba22dba1 P acdb40acba534463877dc53f2dddad1c: Bootstrap starting.
I20260808 22:26:58.480086 13275 tablet_bootstrap.cc:492] T 02a794c13a1741688e1ffc22ba22dba1 P 7517d4c940054cfb948102ef80da83e4: Bootstrap starting.
I20260808 22:26:58.481587 13276 tablet_bootstrap.cc:654] T 02a794c13a1741688e1ffc22ba22dba1 P acdb40acba534463877dc53f2dddad1c: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:58.481586 13274 tablet_bootstrap.cc:654] T 02a794c13a1741688e1ffc22ba22dba1 P bc98f64fcfed4433b0b157fff0f75ffe: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:58.481950 13275 tablet_bootstrap.cc:654] T 02a794c13a1741688e1ffc22ba22dba1 P 7517d4c940054cfb948102ef80da83e4: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:58.482913 13276 log.cc:824] T 02a794c13a1741688e1ffc22ba22dba1 P acdb40acba534463877dc53f2dddad1c: Log is configured to *not* fsync() on all Append() calls
I20260808 22:26:58.483148 13275 log.cc:824] T 02a794c13a1741688e1ffc22ba22dba1 P 7517d4c940054cfb948102ef80da83e4: Log is configured to *not* fsync() on all Append() calls
I20260808 22:26:58.483330 13274 log.cc:824] T 02a794c13a1741688e1ffc22ba22dba1 P bc98f64fcfed4433b0b157fff0f75ffe: Log is configured to *not* fsync() on all Append() calls
I20260808 22:26:58.484987 13276 tablet_bootstrap.cc:492] T 02a794c13a1741688e1ffc22ba22dba1 P acdb40acba534463877dc53f2dddad1c: No bootstrap required, opened a new log
I20260808 22:26:58.485193 13276 ts_tablet_manager.cc:1397] T 02a794c13a1741688e1ffc22ba22dba1 P acdb40acba534463877dc53f2dddad1c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 22:26:58.485225 13274 tablet_bootstrap.cc:492] T 02a794c13a1741688e1ffc22ba22dba1 P bc98f64fcfed4433b0b157fff0f75ffe: No bootstrap required, opened a new log
I20260808 22:26:58.485385 13274 ts_tablet_manager.cc:1397] T 02a794c13a1741688e1ffc22ba22dba1 P bc98f64fcfed4433b0b157fff0f75ffe: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260808 22:26:58.489141 13275 tablet_bootstrap.cc:492] T 02a794c13a1741688e1ffc22ba22dba1 P 7517d4c940054cfb948102ef80da83e4: No bootstrap required, opened a new log
I20260808 22:26:58.489326 13275 ts_tablet_manager.cc:1397] T 02a794c13a1741688e1ffc22ba22dba1 P 7517d4c940054cfb948102ef80da83e4: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260808 22:26:58.489197 13276 raft_consensus.cc:348] T 02a794c13a1741688e1ffc22ba22dba1 P acdb40acba534463877dc53f2dddad1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7517d4c940054cfb948102ef80da83e4" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 44099 } } peers { permanent_uuid: "acdb40acba534463877dc53f2dddad1c" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36643 } } peers { permanent_uuid: "bc98f64fcfed4433b0b157fff0f75ffe" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 36091 } }
I20260808 22:26:58.489292 13274 raft_consensus.cc:348] T 02a794c13a1741688e1ffc22ba22dba1 P bc98f64fcfed4433b0b157fff0f75ffe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7517d4c940054cfb948102ef80da83e4" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 44099 } } peers { permanent_uuid: "acdb40acba534463877dc53f2dddad1c" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36643 } } peers { permanent_uuid: "bc98f64fcfed4433b0b157fff0f75ffe" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 36091 } }
I20260808 22:26:58.489497 13274 raft_consensus.cc:374] T 02a794c13a1741688e1ffc22ba22dba1 P bc98f64fcfed4433b0b157fff0f75ffe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:58.489480 13276 raft_consensus.cc:374] T 02a794c13a1741688e1ffc22ba22dba1 P acdb40acba534463877dc53f2dddad1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:58.489547 13274 raft_consensus.cc:729] T 02a794c13a1741688e1ffc22ba22dba1 P bc98f64fcfed4433b0b157fff0f75ffe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc98f64fcfed4433b0b157fff0f75ffe, State: Initialized, Role: FOLLOWER
I20260808 22:26:58.489554 13276 raft_consensus.cc:729] T 02a794c13a1741688e1ffc22ba22dba1 P acdb40acba534463877dc53f2dddad1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acdb40acba534463877dc53f2dddad1c, State: Initialized, Role: FOLLOWER
I20260808 22:26:58.490026 13274 consensus_queue.cc:260] T 02a794c13a1741688e1ffc22ba22dba1 P bc98f64fcfed4433b0b157fff0f75ffe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7517d4c940054cfb948102ef80da83e4" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 44099 } } peers { permanent_uuid: "acdb40acba534463877dc53f2dddad1c" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36643 } } peers { permanent_uuid: "bc98f64fcfed4433b0b157fff0f75ffe" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 36091 } }
I20260808 22:26:58.490026 13276 consensus_queue.cc:260] T 02a794c13a1741688e1ffc22ba22dba1 P acdb40acba534463877dc53f2dddad1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7517d4c940054cfb948102ef80da83e4" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 44099 } } peers { permanent_uuid: "acdb40acba534463877dc53f2dddad1c" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36643 } } peers { permanent_uuid: "bc98f64fcfed4433b0b157fff0f75ffe" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 36091 } }
I20260808 22:26:58.490729 13281 raft_consensus.cc:482] T 02a794c13a1741688e1ffc22ba22dba1 P bc98f64fcfed4433b0b157fff0f75ffe [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:26:58.490867 13281 raft_consensus.cc:504] T 02a794c13a1741688e1ffc22ba22dba1 P bc98f64fcfed4433b0b157fff0f75ffe [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7517d4c940054cfb948102ef80da83e4" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 44099 } } peers { permanent_uuid: "acdb40acba534463877dc53f2dddad1c" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36643 } } peers { permanent_uuid: "bc98f64fcfed4433b0b157fff0f75ffe" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 36091 } }
I20260808 22:26:58.491015 13276 ts_tablet_manager.cc:1428] T 02a794c13a1741688e1ffc22ba22dba1 P acdb40acba534463877dc53f2dddad1c: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 22:26:58.491025 13274 ts_tablet_manager.cc:1428] T 02a794c13a1741688e1ffc22ba22dba1 P bc98f64fcfed4433b0b157fff0f75ffe: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 22:26:58.491355 12796 heartbeater.cc:503] Master 127.5.11.254:37325 was elected leader, sending a full tablet report...
I20260808 22:26:58.491640 13021 heartbeater.cc:503] Master 127.5.11.254:37325 was elected leader, sending a full tablet report...
I20260808 22:26:58.492173 13281 leader_election.cc:288] T 02a794c13a1741688e1ffc22ba22dba1 P bc98f64fcfed4433b0b157fff0f75ffe [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7517d4c940054cfb948102ef80da83e4 (127.5.11.195:44099), acdb40acba534463877dc53f2dddad1c (127.5.11.194:36643)
I20260808 22:26:58.494575 13275 raft_consensus.cc:348] T 02a794c13a1741688e1ffc22ba22dba1 P 7517d4c940054cfb948102ef80da83e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7517d4c940054cfb948102ef80da83e4" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 44099 } } peers { permanent_uuid: "acdb40acba534463877dc53f2dddad1c" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36643 } } peers { permanent_uuid: "bc98f64fcfed4433b0b157fff0f75ffe" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 36091 } }
I20260808 22:26:58.494925 13275 raft_consensus.cc:374] T 02a794c13a1741688e1ffc22ba22dba1 P 7517d4c940054cfb948102ef80da83e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:58.495019 13275 raft_consensus.cc:729] T 02a794c13a1741688e1ffc22ba22dba1 P 7517d4c940054cfb948102ef80da83e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7517d4c940054cfb948102ef80da83e4, State: Initialized, Role: FOLLOWER
I20260808 22:26:58.495537 13275 consensus_queue.cc:260] T 02a794c13a1741688e1ffc22ba22dba1 P 7517d4c940054cfb948102ef80da83e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7517d4c940054cfb948102ef80da83e4" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 44099 } } peers { permanent_uuid: "acdb40acba534463877dc53f2dddad1c" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36643 } } peers { permanent_uuid: "bc98f64fcfed4433b0b157fff0f75ffe" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 36091 } }
I20260808 22:26:58.496644 13275 ts_tablet_manager.cc:1428] T 02a794c13a1741688e1ffc22ba22dba1 P 7517d4c940054cfb948102ef80da83e4: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260808 22:26:58.497013 12798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:58.492287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:44099 (local address 127.5.11.193:36389)
0808 22:26:58.492458 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:58.492473 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:58.492570 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:58.493089 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:58.493092 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:58.493109 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:26:58.493237 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:58.493244 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:58.494258 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:58.494262 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:58.495493 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:58.495505 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:58.496492 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:58.496496 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:58.496653 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:58.496693 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 22:26:58.496782 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 22:26:58.496866 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":55}
I20260808 22:26:58.497016 13269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:58.492710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:36389 (local address 127.5.11.195:44099)
0808 22:26:58.492824 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:58.492831 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:58.492847 (+ 16us) server_negotiation.cc:407] Connection header received
0808 22:26:58.492890 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:58.492893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:58.492939 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:58.493031 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:26:58.493339 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:58.494118 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:58.495639 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:58.496261 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:58.496377 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:58.496428 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:58.496821 (+ 393us) server_negotiation.cc:299] Negotiation successful
0808 22:26:58.496896 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 22:26:58.497543 13249 heartbeater.cc:503] Master 127.5.11.254:37325 was elected leader, sending a full tablet report...
I20260808 22:26:58.497674 13171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02a794c13a1741688e1ffc22ba22dba1" candidate_uuid: "bc98f64fcfed4433b0b157fff0f75ffe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7517d4c940054cfb948102ef80da83e4" is_pre_election: true
I20260808 22:26:58.497938 13171 raft_consensus.cc:2454] T 02a794c13a1741688e1ffc22ba22dba1 P 7517d4c940054cfb948102ef80da83e4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc98f64fcfed4433b0b157fff0f75ffe in term 0.
I20260808 22:26:58.498445 12612 leader_election.cc:302] T 02a794c13a1741688e1ffc22ba22dba1 P bc98f64fcfed4433b0b157fff0f75ffe [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7517d4c940054cfb948102ef80da83e4, bc98f64fcfed4433b0b157fff0f75ffe; no voters:
I20260808 22:26:58.498775 13273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:58.493458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:38077 (local address 127.5.11.194:36643)
0808 22:26:58.493576 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:58.493580 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:58.493867 (+ 287us) server_negotiation.cc:407] Connection header received
0808 22:26:58.494068 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:58.494071 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:58.494113 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:58.494164 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:26:58.494604 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:58.495294 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:58.497300 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:58.497880 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:58.498004 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:58.498054 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:58.498582 (+ 528us) server_negotiation.cc:299] Negotiation successful
0808 22:26:58.498664 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 22:26:58.498759 13281 raft_consensus.cc:2781] T 02a794c13a1741688e1ffc22ba22dba1 P bc98f64fcfed4433b0b157fff0f75ffe [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:26:58.498833 13281 raft_consensus.cc:482] T 02a794c13a1741688e1ffc22ba22dba1 P bc98f64fcfed4433b0b157fff0f75ffe [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:26:58.498906 13281 raft_consensus.cc:3037] T 02a794c13a1741688e1ffc22ba22dba1 P bc98f64fcfed4433b0b157fff0f75ffe [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:58.499339 13284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:58.492532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:36643 (local address 127.5.11.193:38077)
0808 22:26:58.493781 (+ 1249us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:58.493796 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:58.493931 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:58.494315 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:58.494318 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:58.494334 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:26:58.494452 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:58.494462 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:58.495432 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:58.495435 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:58.497202 (+ 1767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:26:58.497213 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:26:58.498112 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:26:58.498118 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:26:58.498213 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:58.498238 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 22:26:58.499095 (+ 857us) client_negotiation.cc:240] Negotiation successful
0808 22:26:58.499195 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1144,"mutex_wait_us":2,"thread_start_us":88,"threads_started":1}
I20260808 22:26:58.499974 12939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02a794c13a1741688e1ffc22ba22dba1" candidate_uuid: "bc98f64fcfed4433b0b157fff0f75ffe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acdb40acba534463877dc53f2dddad1c" is_pre_election: true
I20260808 22:26:58.500121 13281 raft_consensus.cc:504] T 02a794c13a1741688e1ffc22ba22dba1 P bc98f64fcfed4433b0b157fff0f75ffe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7517d4c940054cfb948102ef80da83e4" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 44099 } } peers { permanent_uuid: "acdb40acba534463877dc53f2dddad1c" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36643 } } peers { permanent_uuid: "bc98f64fcfed4433b0b157fff0f75ffe" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 36091 } }
I20260808 22:26:58.500269 12939 raft_consensus.cc:2454] T 02a794c13a1741688e1ffc22ba22dba1 P acdb40acba534463877dc53f2dddad1c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc98f64fcfed4433b0b157fff0f75ffe in term 0.
I20260808 22:26:58.500689 13281 leader_election.cc:288] T 02a794c13a1741688e1ffc22ba22dba1 P bc98f64fcfed4433b0b157fff0f75ffe [CANDIDATE]: Term 1 election: Requested vote from peers 7517d4c940054cfb948102ef80da83e4 (127.5.11.195:44099), acdb40acba534463877dc53f2dddad1c (127.5.11.194:36643)
I20260808 22:26:58.500877 13171 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02a794c13a1741688e1ffc22ba22dba1" candidate_uuid: "bc98f64fcfed4433b0b157fff0f75ffe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7517d4c940054cfb948102ef80da83e4"
I20260808 22:26:58.501021 13171 raft_consensus.cc:3037] T 02a794c13a1741688e1ffc22ba22dba1 P 7517d4c940054cfb948102ef80da83e4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:58.501160 12939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02a794c13a1741688e1ffc22ba22dba1" candidate_uuid: "bc98f64fcfed4433b0b157fff0f75ffe" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acdb40acba534463877dc53f2dddad1c"
I20260808 22:26:58.501294 12939 raft_consensus.cc:3037] T 02a794c13a1741688e1ffc22ba22dba1 P acdb40acba534463877dc53f2dddad1c [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:58.502138 13171 raft_consensus.cc:2454] T 02a794c13a1741688e1ffc22ba22dba1 P 7517d4c940054cfb948102ef80da83e4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc98f64fcfed4433b0b157fff0f75ffe in term 1.
I20260808 22:26:58.502429 12612 leader_election.cc:302] T 02a794c13a1741688e1ffc22ba22dba1 P bc98f64fcfed4433b0b157fff0f75ffe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7517d4c940054cfb948102ef80da83e4, bc98f64fcfed4433b0b157fff0f75ffe; no voters:
I20260808 22:26:58.502429 12939 raft_consensus.cc:2454] T 02a794c13a1741688e1ffc22ba22dba1 P acdb40acba534463877dc53f2dddad1c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc98f64fcfed4433b0b157fff0f75ffe in term 1.
I20260808 22:26:58.502599 13281 raft_consensus.cc:2781] T 02a794c13a1741688e1ffc22ba22dba1 P bc98f64fcfed4433b0b157fff0f75ffe [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:26:58.502774 13281 raft_consensus.cc:686] T 02a794c13a1741688e1ffc22ba22dba1 P bc98f64fcfed4433b0b157fff0f75ffe [term 1 LEADER]: Becoming Leader. State: Replica: bc98f64fcfed4433b0b157fff0f75ffe, State: Running, Role: LEADER
I20260808 22:26:58.503080 13281 consensus_queue.cc:237] T 02a794c13a1741688e1ffc22ba22dba1 P bc98f64fcfed4433b0b157fff0f75ffe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7517d4c940054cfb948102ef80da83e4" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 44099 } } peers { permanent_uuid: "acdb40acba534463877dc53f2dddad1c" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36643 } } peers { permanent_uuid: "bc98f64fcfed4433b0b157fff0f75ffe" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 36091 } }
I20260808 22:26:58.506091 12489 catalog_manager.cc:5495] T 02a794c13a1741688e1ffc22ba22dba1 P bc98f64fcfed4433b0b157fff0f75ffe reported cstate change: term changed from 0 to 1, leader changed from <none> to bc98f64fcfed4433b0b157fff0f75ffe (127.5.11.193). New cstate: current_term: 1 leader_uuid: "bc98f64fcfed4433b0b157fff0f75ffe" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7517d4c940054cfb948102ef80da83e4" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 44099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "acdb40acba534463877dc53f2dddad1c" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36643 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc98f64fcfed4433b0b157fff0f75ffe" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 36091 } health_report { overall_health: HEALTHY } } }
I20260808 22:26:58.516067 13272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:58.512902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53396 (local address 127.5.11.193:36091)
0808 22:26:58.513015 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:58.513019 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:58.513040 (+ 21us) server_negotiation.cc:407] Connection header received
0808 22:26:58.513170 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:58.513172 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:58.513221 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:58.513307 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:26:58.513535 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:58.513994 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:58.514808 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:26:58.515005 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:26:58.515080 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:26:58.515348 (+ 268us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:26:58.515715 (+ 367us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:26:58.515879 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:58.515926 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 22:26:58.515972 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 22:26:58.521461 12939 raft_consensus.cc:1261] T 02a794c13a1741688e1ffc22ba22dba1 P acdb40acba534463877dc53f2dddad1c [term 1 FOLLOWER]: Refusing update from remote peer bc98f64fcfed4433b0b157fff0f75ffe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:26:58.521462 13171 raft_consensus.cc:1261] T 02a794c13a1741688e1ffc22ba22dba1 P 7517d4c940054cfb948102ef80da83e4 [term 1 FOLLOWER]: Refusing update from remote peer bc98f64fcfed4433b0b157fff0f75ffe: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:26:58.522217 13281 consensus_queue.cc:1035] T 02a794c13a1741688e1ffc22ba22dba1 P bc98f64fcfed4433b0b157fff0f75ffe [LEADER]: Connected to new peer: Peer: permanent_uuid: "acdb40acba534463877dc53f2dddad1c" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:26:58.522464 13280 consensus_queue.cc:1035] T 02a794c13a1741688e1ffc22ba22dba1 P bc98f64fcfed4433b0b157fff0f75ffe [LEADER]: Connected to new peer: Peer: permanent_uuid: "7517d4c940054cfb948102ef80da83e4" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 44099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:26:58.531167 13293 mvcc.cc:205] Tried to move back new op lower bound from 7316389963856982016 to 7316389963791601664. Current Snapshot: MvccSnapshot[applied={T|T < 7316389963856982016}]
I20260808 22:26:58.537706 12488 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:26:58.537905 12488 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:26:58.540349 12488 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1867f778f7d04a969d19293a38bebe6d: Sending DeleteTablet for 3 replicas of tablet 02a794c13a1741688e1ffc22ba22dba1
I20260808 22:26:58.541085 13138 tablet_service.cc:1514] Processing DeleteTablet for tablet 02a794c13a1741688e1ffc22ba22dba1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:26:58 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:37728
I20260808 22:26:58.541253 12905 tablet_service.cc:1514] Processing DeleteTablet for tablet 02a794c13a1741688e1ffc22ba22dba1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:26:58 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:55288
I20260808 22:26:58.541608 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 12591
I20260808 22:26:58.541712 13307 tablet_replica.cc:330] T 02a794c13a1741688e1ffc22ba22dba1 P 7517d4c940054cfb948102ef80da83e4: stopping tablet replica
I20260808 22:26:58.541695 12686 tablet_service.cc:1514] Processing DeleteTablet for tablet 02a794c13a1741688e1ffc22ba22dba1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:26:58 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:53388
I20260808 22:26:58.542227 13307 raft_consensus.cc:2229] T 02a794c13a1741688e1ffc22ba22dba1 P 7517d4c940054cfb948102ef80da83e4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:26:58.542522 13307 raft_consensus.cc:2258] T 02a794c13a1741688e1ffc22ba22dba1 P 7517d4c940054cfb948102ef80da83e4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:26:58.546746 13307 ts_tablet_manager.cc:1905] T 02a794c13a1741688e1ffc22ba22dba1 P 7517d4c940054cfb948102ef80da83e4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:26:58.548920 13309 tablet_replica.cc:330] T 02a794c13a1741688e1ffc22ba22dba1 P acdb40acba534463877dc53f2dddad1c: stopping tablet replica
I20260808 22:26:58.549350 13309 raft_consensus.cc:2229] T 02a794c13a1741688e1ffc22ba22dba1 P acdb40acba534463877dc53f2dddad1c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:26:58.549381 13307 ts_tablet_manager.cc:1918] T 02a794c13a1741688e1ffc22ba22dba1 P 7517d4c940054cfb948102ef80da83e4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:26:58.549499 13307 log.cc:1196] T 02a794c13a1741688e1ffc22ba22dba1 P 7517d4c940054cfb948102ef80da83e4: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-2/wal/wals/02a794c13a1741688e1ffc22ba22dba1
I20260808 22:26:58.549763 13309 raft_consensus.cc:2258] T 02a794c13a1741688e1ffc22ba22dba1 P acdb40acba534463877dc53f2dddad1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:26:58.549868 13307 ts_tablet_manager.cc:1939] T 02a794c13a1741688e1ffc22ba22dba1 P 7517d4c940054cfb948102ef80da83e4: Deleting consensus metadata
I20260808 22:26:58.550976 12464 catalog_manager.cc:4841] TS 7517d4c940054cfb948102ef80da83e4 (127.5.11.195:44099): tablet 02a794c13a1741688e1ffc22ba22dba1 (table test-table [id=93071044b64d45cbbd102ebab1a4bc21]) successfully deleted
I20260808 22:26:58.552809 13309 ts_tablet_manager.cc:1905] T 02a794c13a1741688e1ffc22ba22dba1 P acdb40acba534463877dc53f2dddad1c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:26:58.555032 13309 ts_tablet_manager.cc:1918] T 02a794c13a1741688e1ffc22ba22dba1 P acdb40acba534463877dc53f2dddad1c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:26:58.555147 13309 log.cc:1196] T 02a794c13a1741688e1ffc22ba22dba1 P acdb40acba534463877dc53f2dddad1c: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786227991988852-5167-0/minicluster-data/ts-1/wal/wals/02a794c13a1741688e1ffc22ba22dba1
I20260808 22:26:58.555502 13309 ts_tablet_manager.cc:1939] T 02a794c13a1741688e1ffc22ba22dba1 P acdb40acba534463877dc53f2dddad1c: Deleting consensus metadata
I20260808 22:26:58.556641 12466 catalog_manager.cc:4841] TS acdb40acba534463877dc53f2dddad1c (127.5.11.194:36643): tablet 02a794c13a1741688e1ffc22ba22dba1 (table test-table [id=93071044b64d45cbbd102ebab1a4bc21]) successfully deleted
W20260808 22:26:58.558089 12466 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.11.193:36091 (error 108)
I20260808 22:26:58.558382 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 12803
I20260808 22:26:58.559433 13271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:58.559109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:36091 (local address 127.0.0.1:53400)
0808 22:26:58.559262 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:58.559336 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.193:36091: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 22:26:58.559581 12466 catalog_manager.cc:4584] TS bc98f64fcfed4433b0b157fff0f75ffe (127.5.11.193:36091): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 02a794c13a1741688e1ffc22ba22dba1: Network error: Client connection negotiation failed: client connection to 127.5.11.193:36091: connect: Connection refused (error 111)
I20260808 22:26:58.568150 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 13033
I20260808 22:26:58.578707 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 12432
2026-08-08T22:26:58Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2467 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-08T22:26:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:26:58Z Disabled control of system clock
I20260808 22:26:58.614485 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:37401
--webserver_interface=127.5.11.254
--webserver_port=0
--builtin_ntp_servers=127.5.11.212:36793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:37401
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 22:26:58.726512 13314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:26:58.726820 13314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:26:58.726858 13314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:26:58.730823 13314 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:26:58.730880 13314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:26:58.730895 13314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:26:58.730912 13314 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:26:58.730926 13314 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:26:58.735512 13314 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:36793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:37401
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:37401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13314
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:26:58.736581 13314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:26:58.737901 13314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:26:58.746390 13331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:58.746398 13325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:58.746569 13314 server_base.cc:1034] running on GCE node
W20260808 22:26:58.746485 13323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:58.747359 13314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:26:58.748472 13314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:26:58.749640 13314 hybrid_clock.cc:648] HybridClock initialized: now 1786228018749612 us; error 40 us; skew 500 ppm
I20260808 22:26:58.751403 13314 webserver.cc:460] Webserver started at http://127.5.11.254:40283/ using document root <none> and password file <none>
I20260808 22:26:58.752072 13314 fs_manager.cc:359] Metadata directory not provided
I20260808 22:26:58.752139 13314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:26:58.752462 13314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:26:58.754532 13314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "637b455b6668453fbd937992dd97b129"
format_stamp: "Formatted at 2026-08-08 22:26:58 on dist-test-slave-7f01"
server_key: "0b6de0b3544e538de99c837990f8492c"
server_key_iv: "44176033c890a48aa489d2627c4bbc6e"
server_key_version: "encryptionkey@0"
I20260808 22:26:58.755120 13314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "637b455b6668453fbd937992dd97b129"
format_stamp: "Formatted at 2026-08-08 22:26:58 on dist-test-slave-7f01"
server_key: "0b6de0b3544e538de99c837990f8492c"
server_key_iv: "44176033c890a48aa489d2627c4bbc6e"
server_key_version: "encryptionkey@0"
I20260808 22:26:58.758715 13314 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 22:26:58.761137 13340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:58.762324 13314 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:26:58.762454 13314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "637b455b6668453fbd937992dd97b129"
format_stamp: "Formatted at 2026-08-08 22:26:58 on dist-test-slave-7f01"
server_key: "0b6de0b3544e538de99c837990f8492c"
server_key_iv: "44176033c890a48aa489d2627c4bbc6e"
server_key_version: "encryptionkey@0"
I20260808 22:26:58.762559 13314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:26:58.767086 13314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:26:58.767661 13314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:26:58.767863 13314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:26:58.774700 13314 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:37401
I20260808 22:26:58.774808 13434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:37401 every 8 connection(s)
I20260808 22:26:58.775656 13314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:26:58.778787 13435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:58.779958 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 13314
I20260808 22:26:58.780099 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:26:58.780409 5167 external_mini_cluster.cc:1428] Setting key 2147ca997e6479a7c3b6a953bad26306
I20260808 22:26:58.785317 13435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 637b455b6668453fbd937992dd97b129: Bootstrap starting.
I20260808 22:26:58.785995 13438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:58.781831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36164 (local address 127.5.11.254:37401)
0808 22:26:58.782464 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:58.782483 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:58.782530 (+ 47us) server_negotiation.cc:407] Connection header received
0808 22:26:58.783170 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:58.783185 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:58.783516 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:58.783860 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:58.784702 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:58.784707 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:58.784718 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:26:58.784730 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:58.784787 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:26:58.784802 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:58.784957 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:58.785033 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 22:26:58.785276 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":139,"threads_started":1}
I20260808 22:26:58.787726 13435 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 637b455b6668453fbd937992dd97b129: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:58.788640 13435 log.cc:824] T 00000000000000000000000000000000 P 637b455b6668453fbd937992dd97b129: Log is configured to *not* fsync() on all Append() calls
I20260808 22:26:58.790462 13435 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 637b455b6668453fbd937992dd97b129: No bootstrap required, opened a new log
I20260808 22:26:58.794195 13435 raft_consensus.cc:348] T 00000000000000000000000000000000 P 637b455b6668453fbd937992dd97b129 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "637b455b6668453fbd937992dd97b129" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 37401 } }
I20260808 22:26:58.794404 13435 raft_consensus.cc:374] T 00000000000000000000000000000000 P 637b455b6668453fbd937992dd97b129 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:58.794438 13435 raft_consensus.cc:729] T 00000000000000000000000000000000 P 637b455b6668453fbd937992dd97b129 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 637b455b6668453fbd937992dd97b129, State: Initialized, Role: FOLLOWER
I20260808 22:26:58.794895 13435 consensus_queue.cc:260] T 00000000000000000000000000000000 P 637b455b6668453fbd937992dd97b129 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "637b455b6668453fbd937992dd97b129" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 37401 } }
I20260808 22:26:58.795046 13435 raft_consensus.cc:388] T 00000000000000000000000000000000 P 637b455b6668453fbd937992dd97b129 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:26:58.795133 13435 raft_consensus.cc:482] T 00000000000000000000000000000000 P 637b455b6668453fbd937992dd97b129 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:26:58.795252 13435 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 637b455b6668453fbd937992dd97b129 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:58.796182 13435 raft_consensus.cc:504] T 00000000000000000000000000000000 P 637b455b6668453fbd937992dd97b129 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "637b455b6668453fbd937992dd97b129" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 37401 } }
I20260808 22:26:58.796541 13435 leader_election.cc:302] T 00000000000000000000000000000000 P 637b455b6668453fbd937992dd97b129 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 637b455b6668453fbd937992dd97b129; no voters:
I20260808 22:26:58.796867 13435 leader_election.cc:288] T 00000000000000000000000000000000 P 637b455b6668453fbd937992dd97b129 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:26:58.797004 13442 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 637b455b6668453fbd937992dd97b129 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:26:58.797251 13442 raft_consensus.cc:686] T 00000000000000000000000000000000 P 637b455b6668453fbd937992dd97b129 [term 1 LEADER]: Becoming Leader. State: Replica: 637b455b6668453fbd937992dd97b129, State: Running, Role: LEADER
I20260808 22:26:58.797609 13442 consensus_queue.cc:237] T 00000000000000000000000000000000 P 637b455b6668453fbd937992dd97b129 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "637b455b6668453fbd937992dd97b129" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 37401 } }
I20260808 22:26:58.798210 13435 sys_catalog.cc:564] T 00000000000000000000000000000000 P 637b455b6668453fbd937992dd97b129 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:26:58.799508 13445 sys_catalog.cc:455] T 00000000000000000000000000000000 P 637b455b6668453fbd937992dd97b129 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 637b455b6668453fbd937992dd97b129. Latest consensus state: current_term: 1 leader_uuid: "637b455b6668453fbd937992dd97b129" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "637b455b6668453fbd937992dd97b129" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 37401 } } }
I20260808 22:26:58.799636 13445 sys_catalog.cc:458] T 00000000000000000000000000000000 P 637b455b6668453fbd937992dd97b129 [sys.catalog]: This master's current role is: LEADER
I20260808 22:26:58.800019 13443 sys_catalog.cc:455] T 00000000000000000000000000000000 P 637b455b6668453fbd937992dd97b129 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "637b455b6668453fbd937992dd97b129" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "637b455b6668453fbd937992dd97b129" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 37401 } } }
I20260808 22:26:58.800192 13443 sys_catalog.cc:458] T 00000000000000000000000000000000 P 637b455b6668453fbd937992dd97b129 [sys.catalog]: This master's current role is: LEADER
I20260808 22:26:58.800948 13456 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:26:58.803468 13456 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:26:58.808723 13456 catalog_manager.cc:1347] Generated new cluster ID: e56cb13339cc4b36b726a8da572b1808
I20260808 22:26:58.808790 13456 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:26:58.820544 13456 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:26:58.820854 13456 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:26:58.831988 13456 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 637b455b6668453fbd937992dd97b129: Generated new TSK 0
I20260808 22:26:58.832688 13456 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:26:58.840731 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.193:0
--local_ip_for_outbound_sockets=127.5.11.193
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:37401
--builtin_ntp_servers=127.5.11.212:36793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 22:26:58.957314 13470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:26:58.957597 13470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:26:58.957677 13470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:26:58.961648 13470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:26:58.961722 13470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:26:58.961805 13470 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.193
I20260808 22:26:58.966634 13470 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:36793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:37401
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13470
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.193
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:26:58.967665 13470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:26:58.969056 13470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:26:58.979007 13487 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:58.979065 13470 server_base.cc:1034] running on GCE node
W20260808 22:26:58.979105 13486 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:58.979022 13490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:58.979586 13470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:26:58.980244 13470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:26:58.981438 13470 hybrid_clock.cc:648] HybridClock initialized: now 1786228018981424 us; error 35 us; skew 500 ppm
I20260808 22:26:58.983559 13470 webserver.cc:460] Webserver started at http://127.5.11.193:39613/ using document root <none> and password file <none>
I20260808 22:26:58.984218 13470 fs_manager.cc:359] Metadata directory not provided
I20260808 22:26:58.984315 13470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:26:58.984539 13470 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:26:58.986550 13470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-0/data/instance:
uuid: "b735ba79e3c54af49c37ac3085150496"
format_stamp: "Formatted at 2026-08-08 22:26:58 on dist-test-slave-7f01"
server_key: "df64872621ab12d8d15a1fa78d1d228d"
server_key_iv: "44b5aa2624d15ae620ba7b865a765aab"
server_key_version: "encryptionkey@0"
I20260808 22:26:58.987143 13470 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance:
uuid: "b735ba79e3c54af49c37ac3085150496"
format_stamp: "Formatted at 2026-08-08 22:26:58 on dist-test-slave-7f01"
server_key: "df64872621ab12d8d15a1fa78d1d228d"
server_key_iv: "44b5aa2624d15ae620ba7b865a765aab"
server_key_version: "encryptionkey@0"
I20260808 22:26:58.990850 13470 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 22:26:58.993379 13499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:58.994520 13470 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:26:58.994657 13470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-0/wal
uuid: "b735ba79e3c54af49c37ac3085150496"
format_stamp: "Formatted at 2026-08-08 22:26:58 on dist-test-slave-7f01"
server_key: "df64872621ab12d8d15a1fa78d1d228d"
server_key_iv: "44b5aa2624d15ae620ba7b865a765aab"
server_key_version: "encryptionkey@0"
I20260808 22:26:58.994771 13470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:26:58.999450 13470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:26:59.000165 13470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:26:59.000351 13470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:26:59.000895 13470 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:26:59.001798 13470 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:26:59.001873 13470 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:59.001940 13470 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:26:59.002000 13470 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:59.010687 13470 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.193:33591
I20260808 22:26:59.010768 13680 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.193:33591 every 8 connection(s)
I20260808 22:26:59.011570 13470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
I20260808 22:26:59.017719 13438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:59.013440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:44971 (local address 127.5.11.254:37401)
0808 22:26:59.013583 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:59.013586 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:59.014540 (+ 954us) server_negotiation.cc:407] Connection header received
0808 22:26:59.016007 (+ 1467us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:59.016010 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:59.016057 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:59.016169 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:59.017221 (+ 1052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:59.017225 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:59.017228 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:26:59.017242 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:59.017270 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:26:59.017279 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:59.017377 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:59.017534 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 22:26:59.017581 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 22:26:59.018069 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 13470
I20260808 22:26:59.018218 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance
I20260808 22:26:59.018498 5167 external_mini_cluster.cc:1428] Setting key f54ead0c0b8138f2fb70358da73708a7
I20260808 22:26:59.018770 13685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:59.013742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:37401 (local address 127.5.11.193:44971)
0808 22:26:59.014368 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:59.014406 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:59.015784 (+ 1378us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:59.016345 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:59.016352 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:59.016444 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:59.016965 (+ 521us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:26:59.016966 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:59.017005 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:26:59.017071 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:59.017361 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:59.017371 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:59.017377 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 22:26:59.017578 (+ 201us) client_negotiation.cc:240] Negotiation successful
0808 22:26:59.017886 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":116,"threads_started":1}
I20260808 22:26:59.020092 13681 heartbeater.cc:347] Connected to a master server at 127.5.11.254:37401
I20260808 22:26:59.020323 13681 heartbeater.cc:464] Registering TS with master...
I20260808 22:26:59.020555 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.194:0
--local_ip_for_outbound_sockets=127.5.11.194
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:37401
--builtin_ntp_servers=127.5.11.212:36793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260808 22:26:59.020782 13681 heartbeater.cc:511] Master 127.5.11.254:37401 requested a full tablet report, sending...
I20260808 22:26:59.022249 13373 ts_manager.cc:194] Registered new tserver with Master: b735ba79e3c54af49c37ac3085150496 (127.5.11.193:33591)
W20260808 22:26:59.134477 13687 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:26:59.134719 13687 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:26:59.134756 13687 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:26:59.138711 13687 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:26:59.138760 13687 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:26:59.138813 13687 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.194
I20260808 22:26:59.143303 13687 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:36793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:37401
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13687
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.194
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:26:59.144258 13687 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:26:59.145527 13687 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:26:59.155071 13698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:59.155073 13695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:59.155077 13694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:59.155095 13687 server_base.cc:1034] running on GCE node
I20260808 22:26:59.155880 13687 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:26:59.156498 13687 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:26:59.157716 13687 hybrid_clock.cc:648] HybridClock initialized: now 1786228019157684 us; error 47 us; skew 500 ppm
I20260808 22:26:59.163081 13687 webserver.cc:460] Webserver started at http://127.5.11.194:39561/ using document root <none> and password file <none>
I20260808 22:26:59.163668 13687 fs_manager.cc:359] Metadata directory not provided
I20260808 22:26:59.163805 13687 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:26:59.164036 13687 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:26:59.166194 13687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-1/data/instance:
uuid: "d05a74fa1ba84ef0a141dedaec347f79"
format_stamp: "Formatted at 2026-08-08 22:26:59 on dist-test-slave-7f01"
server_key: "e79391748a4276e9a0c731d8dc477276"
server_key_iv: "0b5b2b0fce68fc69835b59003966389c"
server_key_version: "encryptionkey@0"
I20260808 22:26:59.166817 13687 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance:
uuid: "d05a74fa1ba84ef0a141dedaec347f79"
format_stamp: "Formatted at 2026-08-08 22:26:59 on dist-test-slave-7f01"
server_key: "e79391748a4276e9a0c731d8dc477276"
server_key_iv: "0b5b2b0fce68fc69835b59003966389c"
server_key_version: "encryptionkey@0"
I20260808 22:26:59.170846 13687 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260808 22:26:59.173295 13706 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:59.174515 13687 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:26:59.174649 13687 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-1/wal
uuid: "d05a74fa1ba84ef0a141dedaec347f79"
format_stamp: "Formatted at 2026-08-08 22:26:59 on dist-test-slave-7f01"
server_key: "e79391748a4276e9a0c731d8dc477276"
server_key_iv: "0b5b2b0fce68fc69835b59003966389c"
server_key_version: "encryptionkey@0"
I20260808 22:26:59.174755 13687 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:26:59.179315 13687 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:26:59.179942 13687 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:26:59.180119 13687 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:26:59.180657 13687 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:26:59.181547 13687 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:26:59.181617 13687 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:59.181685 13687 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:26:59.181735 13687 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:59.190052 13687 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.194:35331
I20260808 22:26:59.190135 13892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.194:35331 every 8 connection(s)
I20260808 22:26:59.190941 13687 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
I20260808 22:26:59.196893 13438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:59.192801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:38521 (local address 127.5.11.254:37401)
0808 22:26:59.192936 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:59.192940 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:59.193940 (+ 1000us) server_negotiation.cc:407] Connection header received
0808 22:26:59.195208 (+ 1268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:59.195211 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:59.195252 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:59.195326 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:59.196494 (+ 1168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:59.196497 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:59.196500 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:26:59.196514 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:59.196535 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:26:59.196543 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:59.196622 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:59.196745 (+ 123us) server_negotiation.cc:299] Negotiation successful
0808 22:26:59.196786 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 22:26:59.197836 13896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:59.193142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:37401 (local address 127.5.11.194:38521)
0808 22:26:59.193770 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:59.193843 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:59.195002 (+ 1159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:59.195485 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:59.195491 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:59.195576 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:59.196230 (+ 654us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:26:59.196231 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:59.196270 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:26:59.196342 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:59.196618 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:59.196624 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:59.196629 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:26:59.196755 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 22:26:59.196993 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":120,"threads_started":1}
I20260808 22:26:59.198237 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 13687
I20260808 22:26:59.198387 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance
I20260808 22:26:59.198683 5167 external_mini_cluster.cc:1428] Setting key cdb9bb5ea0685cc38aed1bf2f66d585c
I20260808 22:26:59.199034 13894 heartbeater.cc:347] Connected to a master server at 127.5.11.254:37401
I20260808 22:26:59.199301 13894 heartbeater.cc:464] Registering TS with master...
I20260808 22:26:59.199836 13894 heartbeater.cc:511] Master 127.5.11.254:37401 requested a full tablet report, sending...
I20260808 22:26:59.200915 13373 ts_manager.cc:194] Registered new tserver with Master: d05a74fa1ba84ef0a141dedaec347f79 (127.5.11.194:35331)
I20260808 22:26:59.200999 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.195:0
--local_ip_for_outbound_sockets=127.5.11.195
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:37401
--builtin_ntp_servers=127.5.11.212:36793
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260808 22:26:59.315251 13898 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:26:59.315488 13898 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:26:59.315534 13898 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:26:59.319525 13898 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:26:59.319577 13898 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:26:59.319630 13898 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.195
I20260808 22:26:59.324262 13898 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:36793
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:37401
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13898
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.195
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:26:59.325260 13898 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:26:59.326610 13898 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:26:59.335966 13906 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:59.335959 13905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:59.336468 13908 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:59.337174 13898 server_base.cc:1034] running on GCE node
I20260808 22:26:59.337710 13898 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:26:59.338533 13898 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:26:59.339777 13898 hybrid_clock.cc:648] HybridClock initialized: now 1786228019339735 us; error 47 us; skew 500 ppm
I20260808 22:26:59.342234 13898 webserver.cc:460] Webserver started at http://127.5.11.195:34235/ using document root <none> and password file <none>
I20260808 22:26:59.342988 13898 fs_manager.cc:359] Metadata directory not provided
I20260808 22:26:59.343093 13898 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:26:59.343407 13898 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:26:59.346912 13898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-2/data/instance:
uuid: "760a0a41c79444a4950b890afe622f9a"
format_stamp: "Formatted at 2026-08-08 22:26:59 on dist-test-slave-7f01"
server_key: "8ffff9776a91f424e73451a221b10e41"
server_key_iv: "c0c047ae3b564e0822ad0e482f65462a"
server_key_version: "encryptionkey@0"
I20260808 22:26:59.347616 13898 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance:
uuid: "760a0a41c79444a4950b890afe622f9a"
format_stamp: "Formatted at 2026-08-08 22:26:59 on dist-test-slave-7f01"
server_key: "8ffff9776a91f424e73451a221b10e41"
server_key_iv: "c0c047ae3b564e0822ad0e482f65462a"
server_key_version: "encryptionkey@0"
I20260808 22:26:59.353045 13898 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 22:26:59.357256 13920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:59.358918 13898 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260808 22:26:59.359059 13898 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-2/wal
uuid: "760a0a41c79444a4950b890afe622f9a"
format_stamp: "Formatted at 2026-08-08 22:26:59 on dist-test-slave-7f01"
server_key: "8ffff9776a91f424e73451a221b10e41"
server_key_iv: "c0c047ae3b564e0822ad0e482f65462a"
server_key_version: "encryptionkey@0"
I20260808 22:26:59.359191 13898 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:26:59.367514 13898 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:26:59.368311 13898 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:26:59.368527 13898 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:26:59.369176 13898 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:26:59.370615 13898 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:26:59.370679 13898 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:59.370777 13898 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:26:59.370811 13898 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:26:59.383345 13898 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.195:43761
I20260808 22:26:59.383387 14107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.195:43761 every 8 connection(s)
I20260808 22:26:59.384663 13898 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
I20260808 22:26:59.389051 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 13898
I20260808 22:26:59.389192 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance
I20260808 22:26:59.389464 5167 external_mini_cluster.cc:1428] Setting key a5d5d35d40bbde0ecd1e7b880b9b246b
I20260808 22:26:59.392688 13438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:59.386663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:56281 (local address 127.5.11.254:37401)
0808 22:26:59.386887 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:59.386893 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:59.388434 (+ 1541us) server_negotiation.cc:407] Connection header received
0808 22:26:59.390457 (+ 2023us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:59.390460 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:59.390511 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:59.390585 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:59.392138 (+ 1553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:59.392143 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:59.392146 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:26:59.392161 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:59.392181 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:26:59.392191 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:59.392299 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:59.392509 (+ 210us) server_negotiation.cc:299] Negotiation successful
0808 22:26:59.392551 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 22:26:59.393891 14110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:59.387263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:37401 (local address 127.5.11.195:56281)
0808 22:26:59.388244 (+ 981us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:59.388307 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:59.390218 (+ 1911us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:59.390777 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:59.390788 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:59.390916 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:59.391719 (+ 803us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:26:59.391722 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:59.391820 (+ 98us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:26:59.391920 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:59.392318 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:59.392331 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:59.392339 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 22:26:59.392547 (+ 208us) client_negotiation.cc:240] Negotiation successful
0808 22:26:59.392807 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":181,"threads_started":1}
I20260808 22:26:59.394918 14108 heartbeater.cc:347] Connected to a master server at 127.5.11.254:37401
I20260808 22:26:59.395136 14108 heartbeater.cc:464] Registering TS with master...
I20260808 22:26:59.395543 14108 heartbeater.cc:511] Master 127.5.11.254:37401 requested a full tablet report, sending...
I20260808 22:26:59.396597 13373 ts_manager.cc:194] Registered new tserver with Master: 760a0a41c79444a4950b890afe622f9a (127.5.11.195:43761)
I20260808 22:26:59.403288 5167 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:26:59.406612 13438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:59.405245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36176 (local address 127.5.11.254:37401)
0808 22:26:59.405354 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:59.405358 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:59.405521 (+ 163us) server_negotiation.cc:407] Connection header received
0808 22:26:59.405607 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:59.405609 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:59.405638 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:59.405700 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:59.406322 (+ 622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:59.406325 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:59.406327 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:26:59.406338 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:59.406354 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:26:59.406361 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:59.406427 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:59.406496 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 22:26:59.406525 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 22:26:59.409317 13373 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:36176:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:26:59.412390 13373 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:26:59.424597 14121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:59.421248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:33591 (local address 127.0.0.1:38254)
0808 22:26:59.421961 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:59.421975 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:59.422081 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:59.422774 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:59.422780 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:59.422882 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:59.423634 (+ 752us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:26:59.423639 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:59.423664 (+ 25us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:26:59.423757 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:59.424223 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:59.424232 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:59.424238 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 22:26:59.424359 (+ 121us) client_negotiation.cc:240] Negotiation successful
0808 22:26:59.424431 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"spinlock_wait_cycles":35200,"thread_start_us":110,"threads_started":1}
I20260808 22:26:59.424674 14126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:59.421780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38254 (local address 127.5.11.193:33591)
0808 22:26:59.422238 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:59.422248 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:59.422268 (+ 20us) server_negotiation.cc:407] Connection header received
0808 22:26:59.422311 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:59.422315 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:59.422480 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:59.422622 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:59.423932 (+ 1310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:59.423942 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:59.423958 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:26:59.423975 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:59.424030 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:26:59.424063 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:59.424174 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:59.424394 (+ 220us) server_negotiation.cc:299] Negotiation successful
0808 22:26:59.424479 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":124,"threads_started":1}
I20260808 22:26:59.425073 14128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:59.422294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59288 (local address 127.5.11.194:35331)
0808 22:26:59.422672 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:59.422683 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:59.423280 (+ 597us) server_negotiation.cc:407] Connection header received
0808 22:26:59.423342 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:59.423352 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:59.423472 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:59.423617 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:59.424232 (+ 615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:59.424237 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:59.424248 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:26:59.424258 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:59.424297 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:26:59.424328 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:59.424426 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:59.424802 (+ 376us) server_negotiation.cc:299] Negotiation successful
0808 22:26:59.424882 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":144,"threads_started":1}
I20260808 22:26:59.425293 14125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:59.421766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:35331 (local address 127.0.0.1:59288)
0808 22:26:59.423137 (+ 1371us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:59.423152 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:59.423240 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:59.423604 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:59.423607 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:59.423621 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:59.424096 (+ 475us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:26:59.424098 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:59.424111 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:26:59.424138 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:59.424466 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:59.424469 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:59.424472 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 22:26:59.424553 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 22:26:59.424582 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1311,"thread_start_us":98,"threads_started":1}
I20260808 22:26:59.425523 14124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:59.421226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50018 (local address 127.5.11.195:43761)
0808 22:26:59.421791 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:59.421803 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:59.421979 (+ 176us) server_negotiation.cc:407] Connection header received
0808 22:26:59.422215 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:59.422222 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:59.422422 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:59.422561 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:59.424292 (+ 1731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:59.424298 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:59.424309 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:26:59.424324 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:59.424370 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:26:59.424402 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:59.424500 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:59.424671 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 22:26:59.425271 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":158,"threads_started":1}
I20260808 22:26:59.425964 14123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:59.421064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:43761 (local address 127.0.0.1:50018)
0808 22:26:59.421835 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:59.421887 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:59.422068 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:59.423189 (+ 1121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:59.423192 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:59.423208 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:59.424096 (+ 888us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:26:59.424098 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:59.424110 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:26:59.424138 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:59.424545 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:59.424549 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:59.424555 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 22:26:59.425795 (+ 1240us) client_negotiation.cc:240] Negotiation successful
0808 22:26:59.425835 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":91,"threads_started":1}
I20260808 22:26:59.427954 13576 tablet_service.cc:1467] Processing CreateTablet for tablet e87cf6753be84f378275c9269c95d4f7 (DEFAULT_TABLE table=test-table [id=50e4ff0bccd04eb79043b5d69c3ac3e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:26:59.428416 13995 tablet_service.cc:1467] Processing CreateTablet for tablet e87cf6753be84f378275c9269c95d4f7 (DEFAULT_TABLE table=test-table [id=50e4ff0bccd04eb79043b5d69c3ac3e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:26:59.428454 13792 tablet_service.cc:1467] Processing CreateTablet for tablet e87cf6753be84f378275c9269c95d4f7 (DEFAULT_TABLE table=test-table [id=50e4ff0bccd04eb79043b5d69c3ac3e8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:26:59.429154 13576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e87cf6753be84f378275c9269c95d4f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:59.429426 13792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e87cf6753be84f378275c9269c95d4f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:59.429442 13995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e87cf6753be84f378275c9269c95d4f7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:26:59.435240 14130 tablet_bootstrap.cc:492] T e87cf6753be84f378275c9269c95d4f7 P 760a0a41c79444a4950b890afe622f9a: Bootstrap starting.
I20260808 22:26:59.436723 14131 tablet_bootstrap.cc:492] T e87cf6753be84f378275c9269c95d4f7 P d05a74fa1ba84ef0a141dedaec347f79: Bootstrap starting.
I20260808 22:26:59.437834 14130 tablet_bootstrap.cc:654] T e87cf6753be84f378275c9269c95d4f7 P 760a0a41c79444a4950b890afe622f9a: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:59.438517 14131 tablet_bootstrap.cc:654] T e87cf6753be84f378275c9269c95d4f7 P d05a74fa1ba84ef0a141dedaec347f79: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:59.438557 14130 log.cc:824] T e87cf6753be84f378275c9269c95d4f7 P 760a0a41c79444a4950b890afe622f9a: Log is configured to *not* fsync() on all Append() calls
I20260808 22:26:59.439178 14132 tablet_bootstrap.cc:492] T e87cf6753be84f378275c9269c95d4f7 P b735ba79e3c54af49c37ac3085150496: Bootstrap starting.
I20260808 22:26:59.439348 14131 log.cc:824] T e87cf6753be84f378275c9269c95d4f7 P d05a74fa1ba84ef0a141dedaec347f79: Log is configured to *not* fsync() on all Append() calls
I20260808 22:26:59.440860 14132 tablet_bootstrap.cc:654] T e87cf6753be84f378275c9269c95d4f7 P b735ba79e3c54af49c37ac3085150496: Neither blocks nor log segments found. Creating new log.
I20260808 22:26:59.441047 14131 tablet_bootstrap.cc:492] T e87cf6753be84f378275c9269c95d4f7 P d05a74fa1ba84ef0a141dedaec347f79: No bootstrap required, opened a new log
I20260808 22:26:59.441236 14131 ts_tablet_manager.cc:1397] T e87cf6753be84f378275c9269c95d4f7 P d05a74fa1ba84ef0a141dedaec347f79: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 22:26:59.441598 14132 log.cc:824] T e87cf6753be84f378275c9269c95d4f7 P b735ba79e3c54af49c37ac3085150496: Log is configured to *not* fsync() on all Append() calls
I20260808 22:26:59.443073 14130 tablet_bootstrap.cc:492] T e87cf6753be84f378275c9269c95d4f7 P 760a0a41c79444a4950b890afe622f9a: No bootstrap required, opened a new log
I20260808 22:26:59.443264 14130 ts_tablet_manager.cc:1397] T e87cf6753be84f378275c9269c95d4f7 P 760a0a41c79444a4950b890afe622f9a: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260808 22:26:59.443514 14132 tablet_bootstrap.cc:492] T e87cf6753be84f378275c9269c95d4f7 P b735ba79e3c54af49c37ac3085150496: No bootstrap required, opened a new log
I20260808 22:26:59.443692 14132 ts_tablet_manager.cc:1397] T e87cf6753be84f378275c9269c95d4f7 P b735ba79e3c54af49c37ac3085150496: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 22:26:59.445279 14131 raft_consensus.cc:348] T e87cf6753be84f378275c9269c95d4f7 P d05a74fa1ba84ef0a141dedaec347f79 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "760a0a41c79444a4950b890afe622f9a" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 43761 } } peers { permanent_uuid: "b735ba79e3c54af49c37ac3085150496" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 33591 } } peers { permanent_uuid: "d05a74fa1ba84ef0a141dedaec347f79" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35331 } }
I20260808 22:26:59.445523 14131 raft_consensus.cc:374] T e87cf6753be84f378275c9269c95d4f7 P d05a74fa1ba84ef0a141dedaec347f79 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:59.445577 14131 raft_consensus.cc:729] T e87cf6753be84f378275c9269c95d4f7 P d05a74fa1ba84ef0a141dedaec347f79 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d05a74fa1ba84ef0a141dedaec347f79, State: Initialized, Role: FOLLOWER
I20260808 22:26:59.446028 14131 consensus_queue.cc:260] T e87cf6753be84f378275c9269c95d4f7 P d05a74fa1ba84ef0a141dedaec347f79 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "760a0a41c79444a4950b890afe622f9a" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 43761 } } peers { permanent_uuid: "b735ba79e3c54af49c37ac3085150496" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 33591 } } peers { permanent_uuid: "d05a74fa1ba84ef0a141dedaec347f79" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35331 } }
I20260808 22:26:59.446667 13894 heartbeater.cc:503] Master 127.5.11.254:37401 was elected leader, sending a full tablet report...
I20260808 22:26:59.447799 14130 raft_consensus.cc:348] T e87cf6753be84f378275c9269c95d4f7 P 760a0a41c79444a4950b890afe622f9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "760a0a41c79444a4950b890afe622f9a" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 43761 } } peers { permanent_uuid: "b735ba79e3c54af49c37ac3085150496" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 33591 } } peers { permanent_uuid: "d05a74fa1ba84ef0a141dedaec347f79" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35331 } }
I20260808 22:26:59.448006 14131 ts_tablet_manager.cc:1428] T e87cf6753be84f378275c9269c95d4f7 P d05a74fa1ba84ef0a141dedaec347f79: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 22:26:59.448062 14130 raft_consensus.cc:374] T e87cf6753be84f378275c9269c95d4f7 P 760a0a41c79444a4950b890afe622f9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:59.448117 14130 raft_consensus.cc:729] T e87cf6753be84f378275c9269c95d4f7 P 760a0a41c79444a4950b890afe622f9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 760a0a41c79444a4950b890afe622f9a, State: Initialized, Role: FOLLOWER
I20260808 22:26:59.448064 14132 raft_consensus.cc:348] T e87cf6753be84f378275c9269c95d4f7 P b735ba79e3c54af49c37ac3085150496 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "760a0a41c79444a4950b890afe622f9a" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 43761 } } peers { permanent_uuid: "b735ba79e3c54af49c37ac3085150496" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 33591 } } peers { permanent_uuid: "d05a74fa1ba84ef0a141dedaec347f79" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35331 } }
I20260808 22:26:59.448299 14132 raft_consensus.cc:374] T e87cf6753be84f378275c9269c95d4f7 P b735ba79e3c54af49c37ac3085150496 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:26:59.448376 14132 raft_consensus.cc:729] T e87cf6753be84f378275c9269c95d4f7 P b735ba79e3c54af49c37ac3085150496 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b735ba79e3c54af49c37ac3085150496, State: Initialized, Role: FOLLOWER
I20260808 22:26:59.448575 14130 consensus_queue.cc:260] T e87cf6753be84f378275c9269c95d4f7 P 760a0a41c79444a4950b890afe622f9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "760a0a41c79444a4950b890afe622f9a" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 43761 } } peers { permanent_uuid: "b735ba79e3c54af49c37ac3085150496" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 33591 } } peers { permanent_uuid: "d05a74fa1ba84ef0a141dedaec347f79" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35331 } }
I20260808 22:26:59.448750 14132 consensus_queue.cc:260] T e87cf6753be84f378275c9269c95d4f7 P b735ba79e3c54af49c37ac3085150496 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "760a0a41c79444a4950b890afe622f9a" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 43761 } } peers { permanent_uuid: "b735ba79e3c54af49c37ac3085150496" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 33591 } } peers { permanent_uuid: "d05a74fa1ba84ef0a141dedaec347f79" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35331 } }
I20260808 22:26:59.449271 14108 heartbeater.cc:503] Master 127.5.11.254:37401 was elected leader, sending a full tablet report...
I20260808 22:26:59.449532 14130 ts_tablet_manager.cc:1428] T e87cf6753be84f378275c9269c95d4f7 P 760a0a41c79444a4950b890afe622f9a: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 22:26:59.449545 14132 ts_tablet_manager.cc:1428] T e87cf6753be84f378275c9269c95d4f7 P b735ba79e3c54af49c37ac3085150496: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260808 22:26:59.449841 13681 heartbeater.cc:503] Master 127.5.11.254:37401 was elected leader, sending a full tablet report...
W20260808 22:26:59.512841 13683 tablet.cc:2369] T e87cf6753be84f378275c9269c95d4f7 P b735ba79e3c54af49c37ac3085150496: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 22:26:59.635926 14109 tablet.cc:2369] T e87cf6753be84f378275c9269c95d4f7 P 760a0a41c79444a4950b890afe622f9a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:26:59.647972 14137 raft_consensus.cc:482] T e87cf6753be84f378275c9269c95d4f7 P d05a74fa1ba84ef0a141dedaec347f79 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:26:59.648293 14137 raft_consensus.cc:504] T e87cf6753be84f378275c9269c95d4f7 P d05a74fa1ba84ef0a141dedaec347f79 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "760a0a41c79444a4950b890afe622f9a" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 43761 } } peers { permanent_uuid: "b735ba79e3c54af49c37ac3085150496" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 33591 } } peers { permanent_uuid: "d05a74fa1ba84ef0a141dedaec347f79" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35331 } }
I20260808 22:26:59.649555 14137 leader_election.cc:288] T e87cf6753be84f378275c9269c95d4f7 P d05a74fa1ba84ef0a141dedaec347f79 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 760a0a41c79444a4950b890afe622f9a (127.5.11.195:43761), b735ba79e3c54af49c37ac3085150496 (127.5.11.193:33591)
I20260808 22:26:59.651630 14126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:59.649968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:60115 (local address 127.5.11.193:33591)
0808 22:26:59.650146 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:59.650151 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:59.650179 (+ 28us) server_negotiation.cc:407] Connection header received
0808 22:26:59.650224 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:59.650226 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:59.650259 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:59.650365 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:59.651190 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:59.651194 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:59.651197 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:26:59.651211 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:59.651237 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:26:59.651248 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:59.651345 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:59.651446 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 22:26:59.651489 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260808 22:26:59.651630 14124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:59.649899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:41935 (local address 127.5.11.195:43761)
0808 22:26:59.650040 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:59.650047 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:59.650366 (+ 319us) server_negotiation.cc:407] Connection header received
0808 22:26:59.650449 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:59.650457 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:59.650490 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:59.650560 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:59.651203 (+ 643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:59.651207 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:59.651209 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:26:59.651220 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:59.651238 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:26:59.651248 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:59.651353 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:59.651446 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 22:26:59.651489 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 22:26:59.651635 13896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:59.649832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:33591 (local address 127.5.11.194:60115)
0808 22:26:59.650031 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:59.650048 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:59.650139 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:59.650387 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:59.650393 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:59.650411 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:59.650950 (+ 539us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:26:59.650974 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:59.650989 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:26:59.651024 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:59.651342 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:59.651346 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:59.651349 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 22:26:59.651430 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 22:26:59.651483 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117}
I20260808 22:26:59.651670 14140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:59.649835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:43761 (local address 127.5.11.194:41935)
0808 22:26:59.650256 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:59.650266 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 22:26:59.650371 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:26:59.650623 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:26:59.650626 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:26:59.650640 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:26:59.651038 (+ 398us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:26:59.651040 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:26:59.651050 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:26:59.651075 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:26:59.651342 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:26:59.651346 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:26:59.651349 (+ 3us) client_negotiation.cc:769] Sending connection context
0808 22:26:59.651430 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 22:26:59.651492 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":37,"thread_start_us":142,"threads_started":1}
I20260808 22:26:59.652412 14032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87cf6753be84f378275c9269c95d4f7" candidate_uuid: "d05a74fa1ba84ef0a141dedaec347f79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "760a0a41c79444a4950b890afe622f9a" is_pre_election: true
I20260808 22:26:59.652500 13606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87cf6753be84f378275c9269c95d4f7" candidate_uuid: "d05a74fa1ba84ef0a141dedaec347f79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b735ba79e3c54af49c37ac3085150496" is_pre_election: true
I20260808 22:26:59.652757 14032 raft_consensus.cc:2454] T e87cf6753be84f378275c9269c95d4f7 P 760a0a41c79444a4950b890afe622f9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d05a74fa1ba84ef0a141dedaec347f79 in term 0.
I20260808 22:26:59.652796 13606 raft_consensus.cc:2454] T e87cf6753be84f378275c9269c95d4f7 P b735ba79e3c54af49c37ac3085150496 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d05a74fa1ba84ef0a141dedaec347f79 in term 0.
I20260808 22:26:59.653343 13709 leader_election.cc:302] T e87cf6753be84f378275c9269c95d4f7 P d05a74fa1ba84ef0a141dedaec347f79 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 760a0a41c79444a4950b890afe622f9a, b735ba79e3c54af49c37ac3085150496, d05a74fa1ba84ef0a141dedaec347f79; no voters:
I20260808 22:26:59.653602 14137 raft_consensus.cc:2781] T e87cf6753be84f378275c9269c95d4f7 P d05a74fa1ba84ef0a141dedaec347f79 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:26:59.653689 14137 raft_consensus.cc:482] T e87cf6753be84f378275c9269c95d4f7 P d05a74fa1ba84ef0a141dedaec347f79 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:26:59.653733 14137 raft_consensus.cc:3037] T e87cf6753be84f378275c9269c95d4f7 P d05a74fa1ba84ef0a141dedaec347f79 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:59.654788 14137 raft_consensus.cc:504] T e87cf6753be84f378275c9269c95d4f7 P d05a74fa1ba84ef0a141dedaec347f79 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "760a0a41c79444a4950b890afe622f9a" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 43761 } } peers { permanent_uuid: "b735ba79e3c54af49c37ac3085150496" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 33591 } } peers { permanent_uuid: "d05a74fa1ba84ef0a141dedaec347f79" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35331 } }
I20260808 22:26:59.655135 14137 leader_election.cc:288] T e87cf6753be84f378275c9269c95d4f7 P d05a74fa1ba84ef0a141dedaec347f79 [CANDIDATE]: Term 1 election: Requested vote from peers 760a0a41c79444a4950b890afe622f9a (127.5.11.195:43761), b735ba79e3c54af49c37ac3085150496 (127.5.11.193:33591)
I20260808 22:26:59.655500 14032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87cf6753be84f378275c9269c95d4f7" candidate_uuid: "d05a74fa1ba84ef0a141dedaec347f79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "760a0a41c79444a4950b890afe622f9a"
I20260808 22:26:59.655519 13606 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e87cf6753be84f378275c9269c95d4f7" candidate_uuid: "d05a74fa1ba84ef0a141dedaec347f79" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b735ba79e3c54af49c37ac3085150496"
I20260808 22:26:59.655642 14032 raft_consensus.cc:3037] T e87cf6753be84f378275c9269c95d4f7 P 760a0a41c79444a4950b890afe622f9a [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:59.655616 13606 raft_consensus.cc:3037] T e87cf6753be84f378275c9269c95d4f7 P b735ba79e3c54af49c37ac3085150496 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:26:59.656709 13606 raft_consensus.cc:2454] T e87cf6753be84f378275c9269c95d4f7 P b735ba79e3c54af49c37ac3085150496 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d05a74fa1ba84ef0a141dedaec347f79 in term 1.
I20260808 22:26:59.656715 14032 raft_consensus.cc:2454] T e87cf6753be84f378275c9269c95d4f7 P 760a0a41c79444a4950b890afe622f9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d05a74fa1ba84ef0a141dedaec347f79 in term 1.
I20260808 22:26:59.657047 13713 leader_election.cc:302] T e87cf6753be84f378275c9269c95d4f7 P d05a74fa1ba84ef0a141dedaec347f79 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b735ba79e3c54af49c37ac3085150496, d05a74fa1ba84ef0a141dedaec347f79; no voters:
I20260808 22:26:59.657274 14137 raft_consensus.cc:2781] T e87cf6753be84f378275c9269c95d4f7 P d05a74fa1ba84ef0a141dedaec347f79 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:26:59.657485 14137 raft_consensus.cc:686] T e87cf6753be84f378275c9269c95d4f7 P d05a74fa1ba84ef0a141dedaec347f79 [term 1 LEADER]: Becoming Leader. State: Replica: d05a74fa1ba84ef0a141dedaec347f79, State: Running, Role: LEADER
I20260808 22:26:59.657858 14137 consensus_queue.cc:237] T e87cf6753be84f378275c9269c95d4f7 P d05a74fa1ba84ef0a141dedaec347f79 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "760a0a41c79444a4950b890afe622f9a" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 43761 } } peers { permanent_uuid: "b735ba79e3c54af49c37ac3085150496" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 33591 } } peers { permanent_uuid: "d05a74fa1ba84ef0a141dedaec347f79" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35331 } }
I20260808 22:26:59.661203 13370 catalog_manager.cc:5495] T e87cf6753be84f378275c9269c95d4f7 P d05a74fa1ba84ef0a141dedaec347f79 reported cstate change: term changed from 0 to 1, leader changed from <none> to d05a74fa1ba84ef0a141dedaec347f79 (127.5.11.194). New cstate: current_term: 1 leader_uuid: "d05a74fa1ba84ef0a141dedaec347f79" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "760a0a41c79444a4950b890afe622f9a" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 43761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b735ba79e3c54af49c37ac3085150496" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 33591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d05a74fa1ba84ef0a141dedaec347f79" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35331 } health_report { overall_health: HEALTHY } } }
W20260808 22:26:59.692085 13895 tablet.cc:2369] T e87cf6753be84f378275c9269c95d4f7 P d05a74fa1ba84ef0a141dedaec347f79: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:26:59.716084 14128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:59.714473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59298 (local address 127.5.11.194:35331)
0808 22:26:59.714631 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 22:26:59.714637 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:26:59.714656 (+ 19us) server_negotiation.cc:407] Connection header received
0808 22:26:59.714755 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:26:59.714758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:26:59.714817 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:26:59.714917 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:26:59.715607 (+ 690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:26:59.715611 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:26:59.715614 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:26:59.715625 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:26:59.715656 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:26:59.715664 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:26:59.715786 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0808 22:26:59.715900 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 22:26:59.715941 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 22:26:59.722035 13606 raft_consensus.cc:1261] T e87cf6753be84f378275c9269c95d4f7 P b735ba79e3c54af49c37ac3085150496 [term 1 FOLLOWER]: Refusing update from remote peer d05a74fa1ba84ef0a141dedaec347f79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:26:59.722033 14032 raft_consensus.cc:1261] T e87cf6753be84f378275c9269c95d4f7 P 760a0a41c79444a4950b890afe622f9a [term 1 FOLLOWER]: Refusing update from remote peer d05a74fa1ba84ef0a141dedaec347f79: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:26:59.722827 14142 consensus_queue.cc:1035] T e87cf6753be84f378275c9269c95d4f7 P d05a74fa1ba84ef0a141dedaec347f79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b735ba79e3c54af49c37ac3085150496" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 33591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:26:59.723017 14137 consensus_queue.cc:1035] T e87cf6753be84f378275c9269c95d4f7 P d05a74fa1ba84ef0a141dedaec347f79 [LEADER]: Connected to new peer: Peer: permanent_uuid: "760a0a41c79444a4950b890afe622f9a" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 43761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:26:59.730959 14148 mvcc.cc:205] Tried to move back new op lower bound from 7316389968774094848 to 7316389968521437184. Current Snapshot: MvccSnapshot[applied={T|T < 7316389968774094848}]
I20260808 22:26:59.732029 14149 mvcc.cc:205] Tried to move back new op lower bound from 7316389968774094848 to 7316389968521437184. Current Snapshot: MvccSnapshot[applied={T|T < 7316389968774094848}]
I20260808 22:26:59.739128 13372 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:36176:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:26:59.739264 13372 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:36176:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:26:59.741742 13372 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 637b455b6668453fbd937992dd97b129: Sending DeleteTablet for 3 replicas of tablet e87cf6753be84f378275c9269c95d4f7
I20260808 22:26:59.742570 13576 tablet_service.cc:1514] Processing DeleteTablet for tablet e87cf6753be84f378275c9269c95d4f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:26:59 UTC) from {username='slave'} at 127.0.0.1:38254
I20260808 22:26:59.742728 13792 tablet_service.cc:1514] Processing DeleteTablet for tablet e87cf6753be84f378275c9269c95d4f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:26:59 UTC) from {username='slave'} at 127.0.0.1:59288
I20260808 22:26:59.742726 13995 tablet_service.cc:1514] Processing DeleteTablet for tablet e87cf6753be84f378275c9269c95d4f7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:26:59 UTC) from {username='slave'} at 127.0.0.1:50018
I20260808 22:26:59.743150 14167 tablet_replica.cc:330] T e87cf6753be84f378275c9269c95d4f7 P d05a74fa1ba84ef0a141dedaec347f79: stopping tablet replica
I20260808 22:26:59.743467 14167 raft_consensus.cc:2229] T e87cf6753be84f378275c9269c95d4f7 P d05a74fa1ba84ef0a141dedaec347f79 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:26:59.743552 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 13470
I20260808 22:26:59.748582 14167 raft_consensus.cc:2258] T e87cf6753be84f378275c9269c95d4f7 P d05a74fa1ba84ef0a141dedaec347f79 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:26:59.751260 14167 ts_tablet_manager.cc:1905] T e87cf6753be84f378275c9269c95d4f7 P d05a74fa1ba84ef0a141dedaec347f79: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:26:59.752110 14168 tablet_replica.cc:330] T e87cf6753be84f378275c9269c95d4f7 P 760a0a41c79444a4950b890afe622f9a: stopping tablet replica
I20260808 22:26:59.752498 14168 raft_consensus.cc:2229] T e87cf6753be84f378275c9269c95d4f7 P 760a0a41c79444a4950b890afe622f9a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:26:59.752882 14168 raft_consensus.cc:2258] T e87cf6753be84f378275c9269c95d4f7 P 760a0a41c79444a4950b890afe622f9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:26:59.754765 14167 ts_tablet_manager.cc:1918] T e87cf6753be84f378275c9269c95d4f7 P d05a74fa1ba84ef0a141dedaec347f79: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:26:59.754887 14167 log.cc:1196] T e87cf6753be84f378275c9269c95d4f7 P d05a74fa1ba84ef0a141dedaec347f79: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-1/wal/wals/e87cf6753be84f378275c9269c95d4f7
I20260808 22:26:59.755203 14167 ts_tablet_manager.cc:1939] T e87cf6753be84f378275c9269c95d4f7 P d05a74fa1ba84ef0a141dedaec347f79: Deleting consensus metadata
I20260808 22:26:59.756434 13341 catalog_manager.cc:4841] TS d05a74fa1ba84ef0a141dedaec347f79 (127.5.11.194:35331): tablet e87cf6753be84f378275c9269c95d4f7 (table test-table [id=50e4ff0bccd04eb79043b5d69c3ac3e8]) successfully deleted
I20260808 22:26:59.757335 14168 ts_tablet_manager.cc:1905] T e87cf6753be84f378275c9269c95d4f7 P 760a0a41c79444a4950b890afe622f9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:26:59.759447 14168 ts_tablet_manager.cc:1918] T e87cf6753be84f378275c9269c95d4f7 P 760a0a41c79444a4950b890afe622f9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:26:59.759568 14168 log.cc:1196] T e87cf6753be84f378275c9269c95d4f7 P 760a0a41c79444a4950b890afe622f9a: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786227991988852-5167-0/minicluster-data/ts-2/wal/wals/e87cf6753be84f378275c9269c95d4f7
I20260808 22:26:59.759909 14168 ts_tablet_manager.cc:1939] T e87cf6753be84f378275c9269c95d4f7 P 760a0a41c79444a4950b890afe622f9a: Deleting consensus metadata
I20260808 22:26:59.761018 13341 catalog_manager.cc:4841] TS 760a0a41c79444a4950b890afe622f9a (127.5.11.195:43761): tablet e87cf6753be84f378275c9269c95d4f7 (table test-table [id=50e4ff0bccd04eb79043b5d69c3ac3e8]) successfully deleted
W20260808 22:26:59.761464 13343 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.11.193:33591 (error 108)
I20260808 22:26:59.762387 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 13687
I20260808 22:26:59.762528 14123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:26:59.762203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:33591 (local address 127.0.0.1:38256)
0808 22:26:59.762336 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 22:26:59.762410 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.193:33591: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 22:26:59.767343 13343 catalog_manager.cc:4584] TS b735ba79e3c54af49c37ac3085150496 (127.5.11.193:33591): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e87cf6753be84f378275c9269c95d4f7: Network error: Client connection negotiation failed: client connection to 127.5.11.193:33591: connect: Connection refused (error 111)
I20260808 22:26:59.796866 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 13898
I20260808 22:26:59.812451 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 13314
2026-08-08T22:26:59Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1233 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260808 22:26:59.839656 5167 mini_oidc.cc:152] Starting JWKS server
I20260808 22:26:59.841589 5167 webserver.cc:460] Webserver started at https://127.0.0.1:41297/ using document root <none> and password file <none>
I20260808 22:26:59.841794 5167 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 22:26:59.842083 5167 webserver.cc:460] Webserver started at http://127.0.0.1:34395/ using document root <none> and password file <none>
2026-08-08T22:26:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:26:59Z Disabled control of system clock
I20260808 22:26:59.852391 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:37439
--webserver_interface=127.5.11.254
--webserver_port=0
--builtin_ntp_servers=127.5.11.212:38147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:37439
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41297/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786227991988852-5167-0/testchainca.cert with env {}
W20260808 22:26:59.970808 14182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:26:59.971078 14182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:26:59.971118 14182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:26:59.975136 14182 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:26:59.975190 14182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:26:59.975206 14182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:26:59.975222 14182 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:26:59.975237 14182 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 22:26:59.975298 14182 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 22:26:59.975323 14182 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41297/jwks.json
I20260808 22:26:59.981102 14182 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:38147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:37439
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:37439
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41297/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786227991988852-5167-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14182
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:26:59.982198 14182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:26:59.983235 14182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:26:59.991698 14195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:59.991812 14182 server_base.cc:1034] running on GCE node
W20260808 22:26:59.991709 14198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:26:59.991706 14192 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:26:59.992606 14182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:26:59.993724 14182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:26:59.994917 14182 hybrid_clock.cc:648] HybridClock initialized: now 1786228019994907 us; error 31 us; skew 500 ppm
I20260808 22:26:59.996793 14182 webserver.cc:460] Webserver started at http://127.5.11.254:44435/ using document root <none> and password file <none>
I20260808 22:26:59.997383 14182 fs_manager.cc:359] Metadata directory not provided
I20260808 22:26:59.997476 14182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:26:59.997730 14182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:26:59.999811 14182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "4c1231fcbd374edfb9ab73923f24a9de"
format_stamp: "Formatted at 2026-08-08 22:26:59 on dist-test-slave-7f01"
server_key: "6b7203a027faa3bd223f28cd5477f1f0"
server_key_iv: "8760b99cc2d61c7e948bdd210bf1af2b"
server_key_version: "encryptionkey@0"
I20260808 22:27:00.000401 14182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "4c1231fcbd374edfb9ab73923f24a9de"
format_stamp: "Formatted at 2026-08-08 22:26:59 on dist-test-slave-7f01"
server_key: "6b7203a027faa3bd223f28cd5477f1f0"
server_key_iv: "8760b99cc2d61c7e948bdd210bf1af2b"
server_key_version: "encryptionkey@0"
I20260808 22:27:00.004124 14182 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:27:00.006458 14215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:00.007705 14182 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:27:00.007897 14182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "4c1231fcbd374edfb9ab73923f24a9de"
format_stamp: "Formatted at 2026-08-08 22:26:59 on dist-test-slave-7f01"
server_key: "6b7203a027faa3bd223f28cd5477f1f0"
server_key_iv: "8760b99cc2d61c7e948bdd210bf1af2b"
server_key_version: "encryptionkey@0"
I20260808 22:27:00.008014 14182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:00.032239 14182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:00.032933 14182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:00.033146 14182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:00.040478 14182 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:37439
I20260808 22:27:00.040488 14303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:37439 every 8 connection(s)
I20260808 22:27:00.041416 14182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:27:00.044471 14304 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:27:00.048467 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 14182
I20260808 22:27:00.048611 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:27:00.048902 5167 external_mini_cluster.cc:1428] Setting key 4158298a0dd08997081502e77e5ddbda
I20260808 22:27:00.050611 14304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c1231fcbd374edfb9ab73923f24a9de: Bootstrap starting.
I20260808 22:27:00.052845 14304 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c1231fcbd374edfb9ab73923f24a9de: Neither blocks nor log segments found. Creating new log.
I20260808 22:27:00.053562 14304 log.cc:824] T 00000000000000000000000000000000 P 4c1231fcbd374edfb9ab73923f24a9de: Log is configured to *not* fsync() on all Append() calls
I20260808 22:27:00.055694 14304 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c1231fcbd374edfb9ab73923f24a9de: No bootstrap required, opened a new log
I20260808 22:27:00.058516 14309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:00.050229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57438 (local address 127.5.11.254:37439)
0808 22:27:00.050903 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:00.050926 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:00.050964 (+ 38us) server_negotiation.cc:407] Connection header received
0808 22:27:00.051603 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:00.051619 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:00.052079 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:00.052414 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:00.052845 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:00.053936 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:00.055145 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:00.055600 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:00.055716 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:00.056606 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:00.056610 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:00.056623 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:27:00.056984 (+ 361us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:00.057046 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:27:00.057065 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:00.057143 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:00.057319 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 22:27:00.057602 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":124,"threads_started":1}
I20260808 22:27:00.058987 14304 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c1231fcbd374edfb9ab73923f24a9de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c1231fcbd374edfb9ab73923f24a9de" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 37439 } }
I20260808 22:27:00.059194 14304 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c1231fcbd374edfb9ab73923f24a9de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:27:00.059254 14304 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c1231fcbd374edfb9ab73923f24a9de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c1231fcbd374edfb9ab73923f24a9de, State: Initialized, Role: FOLLOWER
I20260808 22:27:00.059711 14304 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c1231fcbd374edfb9ab73923f24a9de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c1231fcbd374edfb9ab73923f24a9de" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 37439 } }
I20260808 22:27:00.059922 14304 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c1231fcbd374edfb9ab73923f24a9de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:27:00.060014 14304 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c1231fcbd374edfb9ab73923f24a9de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:27:00.060114 14304 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c1231fcbd374edfb9ab73923f24a9de [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:27:00.061092 14304 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c1231fcbd374edfb9ab73923f24a9de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c1231fcbd374edfb9ab73923f24a9de" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 37439 } }
I20260808 22:27:00.061476 5167 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:27:00.061451 14304 leader_election.cc:302] T 00000000000000000000000000000000 P 4c1231fcbd374edfb9ab73923f24a9de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c1231fcbd374edfb9ab73923f24a9de; no voters:
I20260808 22:27:00.061779 14304 leader_election.cc:288] T 00000000000000000000000000000000 P 4c1231fcbd374edfb9ab73923f24a9de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:27:00.061875 14312 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c1231fcbd374edfb9ab73923f24a9de [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:27:00.062093 14312 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c1231fcbd374edfb9ab73923f24a9de [term 1 LEADER]: Becoming Leader. State: Replica: 4c1231fcbd374edfb9ab73923f24a9de, State: Running, Role: LEADER
I20260808 22:27:00.062386 14312 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c1231fcbd374edfb9ab73923f24a9de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c1231fcbd374edfb9ab73923f24a9de" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 37439 } }
I20260808 22:27:00.063210 14304 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c1231fcbd374edfb9ab73923f24a9de [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:27:00.063625 14315 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c1231fcbd374edfb9ab73923f24a9de [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c1231fcbd374edfb9ab73923f24a9de. Latest consensus state: current_term: 1 leader_uuid: "4c1231fcbd374edfb9ab73923f24a9de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c1231fcbd374edfb9ab73923f24a9de" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 37439 } } }
I20260808 22:27:00.063788 14315 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c1231fcbd374edfb9ab73923f24a9de [sys.catalog]: This master's current role is: LEADER
I20260808 22:27:00.064204 14313 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c1231fcbd374edfb9ab73923f24a9de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c1231fcbd374edfb9ab73923f24a9de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c1231fcbd374edfb9ab73923f24a9de" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 37439 } } }
I20260808 22:27:00.064363 14313 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c1231fcbd374edfb9ab73923f24a9de [sys.catalog]: This master's current role is: LEADER
I20260808 22:27:00.064664 14317 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:27:00.067539 14317 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:27:00.072839 14317 catalog_manager.cc:1347] Generated new cluster ID: 29bb1fa1d71f4868ba3dea14ff02ffe9
I20260808 22:27:00.072913 14317 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:27:00.089207 14317 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:27:00.090294 14317 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:27:00.098084 14317 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c1231fcbd374edfb9ab73923f24a9de: Generated new TSK 0
I20260808 22:27:00.098730 14317 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 22:27:00.130515 14349 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260808 22:27:00.130671 14349 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 22:27:00.131263 14309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:00.113150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57444 (local address 127.5.11.254:37439)
0808 22:27:00.113284 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:00.113289 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:00.113452 (+ 163us) server_negotiation.cc:407] Connection header received
0808 22:27:00.113640 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:00.113644 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:00.113699 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:00.113804 (+ 105us) server_negotiation.cc:226] Negotiated authn=JWT
0808 22:27:00.114029 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:00.114870 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:00.115707 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:00.116078 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:00.116199 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:00.116501 (+ 302us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 22:27:00.130847 (+ 14346us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41297/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 22:27:00.131110 (+ 263us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57444: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41297/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 22:27:00.131350 14309 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57444: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41297/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260808 22:27:00.132550 14348 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:00.113051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:37439 (local address 127.0.0.1:57444)
0808 22:27:00.113375 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:00.113388 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:00.113495 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:00.113820 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:00.113822 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:00.113833 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0808 22:27:00.113930 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:00.113935 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:00.114970 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:00.114973 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:00.115596 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:00.115604 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:00.116206 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:00.116210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:00.116280 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:00.116322 (+ 42us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 22:27:00.131068 (+ 14746us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41297/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0808 22:27:00.131242 (+ 174us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.11.254:37439: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41297/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":105,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.5.11.254:37439: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41297/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260808 22:27:00.133232 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 14182
2026-08-08T22:27:00Z chronyd exiting
I20260808 22:27:00.176415 5167 test_util.cc:175] -----------------------------------------------
I20260808 22:27:00.176537 5167 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786227991988852-5167-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (338 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260808 22:27:00.177786 5167 mini_oidc.cc:152] Starting JWKS server
I20260808 22:27:00.178668 5167 webserver.cc:460] Webserver started at https://127.0.0.1:46681/ using document root <none> and password file <none>
I20260808 22:27:00.178803 5167 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 22:27:00.179075 5167 webserver.cc:460] Webserver started at http://127.0.0.1:41141/ using document root <none> and password file <none>
2026-08-08T22:27:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:27:00Z Disabled control of system clock
I20260808 22:27:00.189219 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:43015
--webserver_interface=127.5.11.254
--webserver_port=0
--builtin_ntp_servers=127.5.11.212:38283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:43015
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46681/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786227991988852-5167-0/testchainca.cert with env {}
W20260808 22:27:00.300232 14363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:00.300485 14363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:00.300534 14363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:00.304472 14363 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:27:00.304529 14363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:00.304545 14363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:00.304561 14363 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:27:00.304612 14363 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 22:27:00.304633 14363 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 22:27:00.304668 14363 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46681/jwks.json
I20260808 22:27:00.309957 14363 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:38283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:43015
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:43015
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46681/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786227991988852-5167-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:00.310954 14363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:00.311993 14363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:00.320197 14369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:00.320241 14378 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:00.320241 14370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:00.320586 14363 server_base.cc:1034] running on GCE node
I20260808 22:27:00.321130 14363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:00.322186 14363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:00.323381 14363 hybrid_clock.cc:648] HybridClock initialized: now 1786228020323364 us; error 37 us; skew 500 ppm
I20260808 22:27:00.325290 14363 webserver.cc:460] Webserver started at http://127.5.11.254:35565/ using document root <none> and password file <none>
I20260808 22:27:00.325866 14363 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:00.325963 14363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:00.326193 14363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:27:00.328270 14363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "fcb492facbbe42c1bfc5d405d06ed1b7"
format_stamp: "Formatted at 2026-08-08 22:27:00 on dist-test-slave-7f01"
server_key: "8199050e9837c2f2dcc0eb9d32229904"
server_key_iv: "4c9c0c43532035a55e0392cec5004cb7"
server_key_version: "encryptionkey@0"
I20260808 22:27:00.328850 14363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "fcb492facbbe42c1bfc5d405d06ed1b7"
format_stamp: "Formatted at 2026-08-08 22:27:00 on dist-test-slave-7f01"
server_key: "8199050e9837c2f2dcc0eb9d32229904"
server_key_iv: "4c9c0c43532035a55e0392cec5004cb7"
server_key_version: "encryptionkey@0"
I20260808 22:27:00.332471 14363 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 22:27:00.334780 14392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:00.336092 14363 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:27:00.336191 14363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "fcb492facbbe42c1bfc5d405d06ed1b7"
format_stamp: "Formatted at 2026-08-08 22:27:00 on dist-test-slave-7f01"
server_key: "8199050e9837c2f2dcc0eb9d32229904"
server_key_iv: "4c9c0c43532035a55e0392cec5004cb7"
server_key_version: "encryptionkey@0"
I20260808 22:27:00.336274 14363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:00.350864 14363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:00.351420 14363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:00.351565 14363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:00.358424 14363 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:43015
I20260808 22:27:00.358431 14481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:43015 every 8 connection(s)
I20260808 22:27:00.359356 14363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:27:00.362401 14482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:27:00.365182 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 14363
I20260808 22:27:00.365314 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:27:00.365636 5167 external_mini_cluster.cc:1428] Setting key abb32f24b21de8d8f6eac1b71808b32e
I20260808 22:27:00.368660 14482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcb492facbbe42c1bfc5d405d06ed1b7: Bootstrap starting.
I20260808 22:27:00.370798 14482 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fcb492facbbe42c1bfc5d405d06ed1b7: Neither blocks nor log segments found. Creating new log.
I20260808 22:27:00.371536 14482 log.cc:824] T 00000000000000000000000000000000 P fcb492facbbe42c1bfc5d405d06ed1b7: Log is configured to *not* fsync() on all Append() calls
I20260808 22:27:00.373349 14482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcb492facbbe42c1bfc5d405d06ed1b7: No bootstrap required, opened a new log
I20260808 22:27:00.374683 14488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:00.367021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55980 (local address 127.5.11.254:43015)
0808 22:27:00.367677 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:00.367691 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:00.367726 (+ 35us) server_negotiation.cc:407] Connection header received
0808 22:27:00.368349 (+ 623us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:00.368366 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:00.368752 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:00.369118 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:00.369557 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:00.370479 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:00.371431 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:00.372008 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:00.372185 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:00.373047 (+ 862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:00.373053 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:00.373066 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:27:00.373425 (+ 359us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:00.373530 (+ 105us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:27:00.373553 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:00.373658 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:00.373775 (+ 117us) server_negotiation.cc:299] Negotiation successful
0808 22:27:00.374028 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":136,"threads_started":1}
I20260808 22:27:00.377357 14482 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcb492facbbe42c1bfc5d405d06ed1b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb492facbbe42c1bfc5d405d06ed1b7" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 43015 } }
I20260808 22:27:00.377637 14482 raft_consensus.cc:374] T 00000000000000000000000000000000 P fcb492facbbe42c1bfc5d405d06ed1b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:27:00.377729 14482 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcb492facbbe42c1bfc5d405d06ed1b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcb492facbbe42c1bfc5d405d06ed1b7, State: Initialized, Role: FOLLOWER
I20260808 22:27:00.377722 5167 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:27:00.378197 14482 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcb492facbbe42c1bfc5d405d06ed1b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb492facbbe42c1bfc5d405d06ed1b7" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 43015 } }
I20260808 22:27:00.378402 14482 raft_consensus.cc:388] T 00000000000000000000000000000000 P fcb492facbbe42c1bfc5d405d06ed1b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:27:00.378475 14482 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcb492facbbe42c1bfc5d405d06ed1b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:27:00.378576 14482 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcb492facbbe42c1bfc5d405d06ed1b7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:27:00.379720 14482 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcb492facbbe42c1bfc5d405d06ed1b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb492facbbe42c1bfc5d405d06ed1b7" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 43015 } }
I20260808 22:27:00.380075 14482 leader_election.cc:302] T 00000000000000000000000000000000 P fcb492facbbe42c1bfc5d405d06ed1b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fcb492facbbe42c1bfc5d405d06ed1b7; no voters:
I20260808 22:27:00.380338 14482 leader_election.cc:288] T 00000000000000000000000000000000 P fcb492facbbe42c1bfc5d405d06ed1b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:27:00.380476 14491 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fcb492facbbe42c1bfc5d405d06ed1b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:27:00.380717 14491 raft_consensus.cc:686] T 00000000000000000000000000000000 P fcb492facbbe42c1bfc5d405d06ed1b7 [term 1 LEADER]: Becoming Leader. State: Replica: fcb492facbbe42c1bfc5d405d06ed1b7, State: Running, Role: LEADER
I20260808 22:27:00.381031 14491 consensus_queue.cc:237] T 00000000000000000000000000000000 P fcb492facbbe42c1bfc5d405d06ed1b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb492facbbe42c1bfc5d405d06ed1b7" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 43015 } }
I20260808 22:27:00.381513 14482 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcb492facbbe42c1bfc5d405d06ed1b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:27:00.382671 14492 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcb492facbbe42c1bfc5d405d06ed1b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fcb492facbbe42c1bfc5d405d06ed1b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb492facbbe42c1bfc5d405d06ed1b7" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 43015 } } }
I20260808 22:27:00.382805 14492 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcb492facbbe42c1bfc5d405d06ed1b7 [sys.catalog]: This master's current role is: LEADER
I20260808 22:27:00.383129 14493 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcb492facbbe42c1bfc5d405d06ed1b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fcb492facbbe42c1bfc5d405d06ed1b7. Latest consensus state: current_term: 1 leader_uuid: "fcb492facbbe42c1bfc5d405d06ed1b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcb492facbbe42c1bfc5d405d06ed1b7" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 43015 } } }
I20260808 22:27:00.383221 14501 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:27:00.383237 14493 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcb492facbbe42c1bfc5d405d06ed1b7 [sys.catalog]: This master's current role is: LEADER
I20260808 22:27:00.386111 14501 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:27:00.391325 14501 catalog_manager.cc:1347] Generated new cluster ID: 227d53f2e03849c6b3eb63e682ae3bad
I20260808 22:27:00.391390 14501 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:27:00.410590 14501 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:27:00.411886 14501 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:27:00.417608 14501 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fcb492facbbe42c1bfc5d405d06ed1b7: Generated new TSK 0
I20260808 22:27:00.418393 14501 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260808 22:27:00.453090 14528 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260808 22:27:00.453184 14528 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260808 22:27:00.453776 14488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:00.430480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55984 (local address 127.5.11.254:43015)
0808 22:27:00.430624 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:00.430631 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:00.430798 (+ 167us) server_negotiation.cc:407] Connection header received
0808 22:27:00.430961 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:00.430963 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:00.431014 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:00.431100 (+ 86us) server_negotiation.cc:226] Negotiated authn=JWT
0808 22:27:00.431373 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:00.431957 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:00.433209 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:00.433424 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:00.433498 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:00.433829 (+ 331us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 22:27:00.453425 (+ 19596us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46681/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 22:27:00.453652 (+ 227us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55984: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46681/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":57}
W20260808 22:27:00.453867 14488 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55984: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46681/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260808 22:27:00.453975 14526 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:00.430374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43015 (local address 127.0.0.1:55984)
0808 22:27:00.430703 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:00.430719 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:00.430840 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:00.431129 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:00.431133 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:00.431141 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0808 22:27:00.431263 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:00.431269 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:00.432084 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:00.432088 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:00.433094 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:00.433104 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:00.433534 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:00.433538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:00.433631 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:00.433660 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 22:27:00.453617 (+ 19957us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46681/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0808 22:27:00.453794 (+ 177us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.11.254:43015: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46681/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":128,"threads_started":1}
I20260808 22:27:00.454645 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 14363
2026-08-08T22:27:00Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (327 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260808 22:27:00.505769 5167 mini_oidc.cc:152] Starting JWKS server
I20260808 22:27:00.506682 5167 webserver.cc:460] Webserver started at https://127.0.0.1:44805/ using document root <none> and password file <none>
I20260808 22:27:00.506814 5167 mini_oidc.cc:189] Starting OIDC Discovery server
I20260808 22:27:00.507144 5167 webserver.cc:460] Webserver started at http://127.0.0.1:37023/ using document root <none> and password file <none>
2026-08-08T22:27:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:27:00Z Disabled control of system clock
I20260808 22:27:00.517760 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:35891
--webserver_interface=127.5.11.254
--webserver_port=0
--builtin_ntp_servers=127.5.11.212:41931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:35891
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44805/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 22:27:00.628984 14541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:00.629251 14541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:00.629288 14541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:00.633165 14541 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:27:00.633209 14541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:00.633225 14541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:00.633241 14541 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:27:00.633256 14541 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 22:27:00.633284 14541 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 22:27:00.633297 14541 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44805/jwks.json
I20260808 22:27:00.637771 14541 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:41931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:35891
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:35891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44805/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:00.638684 14541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:00.639969 14541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:00.648108 14550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:00.648126 14558 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:00.648121 14551 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:00.648576 14541 server_base.cc:1034] running on GCE node
I20260808 22:27:00.649145 14541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:00.650192 14541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:00.651384 14541 hybrid_clock.cc:648] HybridClock initialized: now 1786228020651375 us; error 28 us; skew 500 ppm
I20260808 22:27:00.653249 14541 webserver.cc:460] Webserver started at http://127.5.11.254:39631/ using document root <none> and password file <none>
I20260808 22:27:00.653838 14541 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:00.653931 14541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:00.654182 14541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:27:00.656286 14541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "fd46deb7a92c4c0a9c914ff23b196670"
format_stamp: "Formatted at 2026-08-08 22:27:00 on dist-test-slave-7f01"
server_key: "d7c2c770dd3fd46958f24e6ec38ba18c"
server_key_iv: "85a74108c39504ebdbc851fcea0aa214"
server_key_version: "encryptionkey@0"
I20260808 22:27:00.656863 14541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "fd46deb7a92c4c0a9c914ff23b196670"
format_stamp: "Formatted at 2026-08-08 22:27:00 on dist-test-slave-7f01"
server_key: "d7c2c770dd3fd46958f24e6ec38ba18c"
server_key_iv: "85a74108c39504ebdbc851fcea0aa214"
server_key_version: "encryptionkey@0"
I20260808 22:27:00.660461 14541 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 22:27:00.662889 14568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:00.664126 14541 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260808 22:27:00.664269 14541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "fd46deb7a92c4c0a9c914ff23b196670"
format_stamp: "Formatted at 2026-08-08 22:27:00 on dist-test-slave-7f01"
server_key: "d7c2c770dd3fd46958f24e6ec38ba18c"
server_key_iv: "85a74108c39504ebdbc851fcea0aa214"
server_key_version: "encryptionkey@0"
I20260808 22:27:00.664382 14541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:00.681704 14541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:00.682341 14541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:00.682561 14541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:00.689184 14541 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:35891
I20260808 22:27:00.689193 14669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:35891 every 8 connection(s)
I20260808 22:27:00.690088 14541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:27:00.693332 14671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:27:00.693719 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 14541
I20260808 22:27:00.693856 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:27:00.694136 5167 external_mini_cluster.cc:1428] Setting key fde8ed5af715fe4372d86444e9a18ba6
I20260808 22:27:00.699229 14671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd46deb7a92c4c0a9c914ff23b196670: Bootstrap starting.
I20260808 22:27:00.701638 14671 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd46deb7a92c4c0a9c914ff23b196670: Neither blocks nor log segments found. Creating new log.
I20260808 22:27:00.702399 14671 log.cc:824] T 00000000000000000000000000000000 P fd46deb7a92c4c0a9c914ff23b196670: Log is configured to *not* fsync() on all Append() calls
I20260808 22:27:00.703509 14675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:00.695536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52736 (local address 127.5.11.254:35891)
0808 22:27:00.696184 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:00.696199 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:00.696234 (+ 35us) server_negotiation.cc:407] Connection header received
0808 22:27:00.696881 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:00.696897 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:00.697282 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:00.697624 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:00.698064 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:00.699000 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:00.700026 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:00.700497 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:00.700678 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:00.701546 (+ 868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:00.701552 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:00.701564 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:27:00.701999 (+ 435us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:00.702055 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:27:00.702076 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:00.702187 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:00.702348 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 22:27:00.702652 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":132,"threads_started":1}
I20260808 22:27:00.704464 14671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd46deb7a92c4c0a9c914ff23b196670: No bootstrap required, opened a new log
I20260808 22:27:00.706453 5167 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:27:00.708771 14671 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd46deb7a92c4c0a9c914ff23b196670 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd46deb7a92c4c0a9c914ff23b196670" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 35891 } }
I20260808 22:27:00.708983 14671 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd46deb7a92c4c0a9c914ff23b196670 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:27:00.709069 14671 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd46deb7a92c4c0a9c914ff23b196670 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd46deb7a92c4c0a9c914ff23b196670, State: Initialized, Role: FOLLOWER
I20260808 22:27:00.709542 14671 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd46deb7a92c4c0a9c914ff23b196670 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd46deb7a92c4c0a9c914ff23b196670" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 35891 } }
I20260808 22:27:00.709692 14671 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd46deb7a92c4c0a9c914ff23b196670 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:27:00.709760 14671 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd46deb7a92c4c0a9c914ff23b196670 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:27:00.709858 14671 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd46deb7a92c4c0a9c914ff23b196670 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:27:00.710827 14671 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd46deb7a92c4c0a9c914ff23b196670 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd46deb7a92c4c0a9c914ff23b196670" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 35891 } }
I20260808 22:27:00.711225 14671 leader_election.cc:302] T 00000000000000000000000000000000 P fd46deb7a92c4c0a9c914ff23b196670 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd46deb7a92c4c0a9c914ff23b196670; no voters:
I20260808 22:27:00.711565 14671 leader_election.cc:288] T 00000000000000000000000000000000 P fd46deb7a92c4c0a9c914ff23b196670 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:27:00.711684 14678 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd46deb7a92c4c0a9c914ff23b196670 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:27:00.711954 14678 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd46deb7a92c4c0a9c914ff23b196670 [term 1 LEADER]: Becoming Leader. State: Replica: fd46deb7a92c4c0a9c914ff23b196670, State: Running, Role: LEADER
I20260808 22:27:00.712272 14678 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd46deb7a92c4c0a9c914ff23b196670 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd46deb7a92c4c0a9c914ff23b196670" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 35891 } }
I20260808 22:27:00.712795 14671 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd46deb7a92c4c0a9c914ff23b196670 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:27:00.714035 14682 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd46deb7a92c4c0a9c914ff23b196670 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd46deb7a92c4c0a9c914ff23b196670. Latest consensus state: current_term: 1 leader_uuid: "fd46deb7a92c4c0a9c914ff23b196670" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd46deb7a92c4c0a9c914ff23b196670" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 35891 } } }
I20260808 22:27:00.714212 14682 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd46deb7a92c4c0a9c914ff23b196670 [sys.catalog]: This master's current role is: LEADER
I20260808 22:27:00.714041 14681 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd46deb7a92c4c0a9c914ff23b196670 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd46deb7a92c4c0a9c914ff23b196670" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd46deb7a92c4c0a9c914ff23b196670" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 35891 } } }
I20260808 22:27:00.714268 14681 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd46deb7a92c4c0a9c914ff23b196670 [sys.catalog]: This master's current role is: LEADER
I20260808 22:27:00.714625 14693 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:27:00.717854 14693 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:27:00.723685 14693 catalog_manager.cc:1347] Generated new cluster ID: 7b6c417cdffe424189f9bf34f005130b
I20260808 22:27:00.723773 14693 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:27:00.730641 14693 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:27:00.731848 14693 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:27:00.738919 14693 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd46deb7a92c4c0a9c914ff23b196670: Generated new TSK 0
I20260808 22:27:00.739657 14693 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:27:00.759450 14675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:00.754333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52738 (local address 127.5.11.254:35891)
0808 22:27:00.754473 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:00.754479 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:00.754631 (+ 152us) server_negotiation.cc:407] Connection header received
0808 22:27:00.754761 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:00.754765 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:00.754821 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:00.754939 (+ 118us) server_negotiation.cc:226] Negotiated authn=JWT
0808 22:27:00.755171 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:00.756026 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:00.756868 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:00.757220 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:00.757322 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:00.757604 (+ 282us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 22:27:00.759070 (+ 1466us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44805/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 22:27:00.759300 (+ 230us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52738: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44805/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 22:27:00.759531 14675 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52738: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44805/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260808 22:27:00.761121 14719 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:00.754192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:35891 (local address 127.0.0.1:52738)
0808 22:27:00.754544 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:00.754557 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:00.754654 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:00.754942 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:00.754944 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:00.754950 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0808 22:27:00.755068 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:00.755073 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:00.756136 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:00.756139 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:00.756754 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:00.756761 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:00.757343 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:00.757346 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:00.757408 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:00.757430 (+ 22us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 22:27:00.759242 (+ 1812us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44805/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0808 22:27:00.760934 (+ 1692us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.11.254:35891: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44805/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":1507,"thread_start_us":122,"threads_started":1}
I20260808 22:27:00.762041 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 14541
E20260808 22:27:00.772507 14720 webserver.cc:533] Webserver: sslize failed:
E20260808 22:27:00.772578 14720 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-08T22:27:00Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (300 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-08T22:27:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:27:00Z Disabled control of system clock
I20260808 22:27:00.815682 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:36503
--webserver_interface=127.5.11.254
--webserver_port=0
--builtin_ntp_servers=127.5.11.212:41289
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:36503
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786227991988852-5167-0/testchainca.cert with env {}
W20260808 22:27:00.928560 14726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:00.928820 14726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:00.928887 14726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:00.932898 14726 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:27:00.932950 14726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:00.932978 14726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:00.932994 14726 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:27:00.933027 14726 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260808 22:27:00.933043 14726 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260808 22:27:00.933056 14726 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260808 22:27:00.938140 14726 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:41289
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:36503
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:36503
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786227991988852-5167-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14726
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:00.939149 14726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:00.940157 14726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:00.949157 14741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:00.949546 14737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:00.949945 14726 server_base.cc:1034] running on GCE node
W20260808 22:27:00.950616 14739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:00.951215 14726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:00.952360 14726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:00.953567 14726 hybrid_clock.cc:648] HybridClock initialized: now 1786228020953538 us; error 46 us; skew 500 ppm
I20260808 22:27:00.955428 14726 webserver.cc:460] Webserver started at http://127.5.11.254:44789/ using document root <none> and password file <none>
I20260808 22:27:00.956080 14726 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:00.956178 14726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:00.956413 14726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:27:00.958433 14726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "be6ac173201549288841721ae3b4695a"
format_stamp: "Formatted at 2026-08-08 22:27:00 on dist-test-slave-7f01"
server_key: "d24bdc9687f30e33c9b4311f870fa908"
server_key_iv: "517aa675dd41c48b64fcbc8b90b8cf98"
server_key_version: "encryptionkey@0"
I20260808 22:27:00.959012 14726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "be6ac173201549288841721ae3b4695a"
format_stamp: "Formatted at 2026-08-08 22:27:00 on dist-test-slave-7f01"
server_key: "d24bdc9687f30e33c9b4311f870fa908"
server_key_iv: "517aa675dd41c48b64fcbc8b90b8cf98"
server_key_version: "encryptionkey@0"
I20260808 22:27:00.962742 14726 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.003s
I20260808 22:27:00.965185 14751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:00.966336 14726 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:27:00.966487 14726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "be6ac173201549288841721ae3b4695a"
format_stamp: "Formatted at 2026-08-08 22:27:00 on dist-test-slave-7f01"
server_key: "d24bdc9687f30e33c9b4311f870fa908"
server_key_iv: "517aa675dd41c48b64fcbc8b90b8cf98"
server_key_version: "encryptionkey@0"
I20260808 22:27:00.966603 14726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:00.980763 14726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:00.981410 14726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:00.981607 14726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:00.988724 14726 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:36503
I20260808 22:27:00.988750 14837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:36503 every 8 connection(s)
I20260808 22:27:00.989667 14726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:27:00.991469 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 14726
I20260808 22:27:00.991600 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:27:00.991968 5167 external_mini_cluster.cc:1428] Setting key f861f6bcadd92419e39e1b35ad258322
I20260808 22:27:00.993499 14839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:27:00.999192 14839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be6ac173201549288841721ae3b4695a: Bootstrap starting.
I20260808 22:27:01.001216 14842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:00.993309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46384 (local address 127.5.11.254:36503)
0808 22:27:00.994003 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:00.994019 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:00.994054 (+ 35us) server_negotiation.cc:407] Connection header received
0808 22:27:00.994683 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:00.994699 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:00.995084 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:00.995434 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:00.995945 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:00.996902 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:00.997827 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:00.998297 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:00.998473 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:00.999410 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:00.999415 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:00.999427 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:27:00.999868 (+ 441us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:00.999928 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:27:00.999951 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:01.000089 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:01.000261 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 22:27:01.000520 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":157,"threads_started":1}
I20260808 22:27:01.001657 14839 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P be6ac173201549288841721ae3b4695a: Neither blocks nor log segments found. Creating new log.
I20260808 22:27:01.002417 14839 log.cc:824] T 00000000000000000000000000000000 P be6ac173201549288841721ae3b4695a: Log is configured to *not* fsync() on all Append() calls
I20260808 22:27:01.004287 5167 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:27:01.004668 14839 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P be6ac173201549288841721ae3b4695a: No bootstrap required, opened a new log
I20260808 22:27:01.008366 14839 raft_consensus.cc:348] T 00000000000000000000000000000000 P be6ac173201549288841721ae3b4695a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be6ac173201549288841721ae3b4695a" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 36503 } }
I20260808 22:27:01.008594 14839 raft_consensus.cc:374] T 00000000000000000000000000000000 P be6ac173201549288841721ae3b4695a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:27:01.008679 14839 raft_consensus.cc:729] T 00000000000000000000000000000000 P be6ac173201549288841721ae3b4695a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be6ac173201549288841721ae3b4695a, State: Initialized, Role: FOLLOWER
I20260808 22:27:01.009132 14839 consensus_queue.cc:260] T 00000000000000000000000000000000 P be6ac173201549288841721ae3b4695a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be6ac173201549288841721ae3b4695a" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 36503 } }
I20260808 22:27:01.009294 14839 raft_consensus.cc:388] T 00000000000000000000000000000000 P be6ac173201549288841721ae3b4695a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:27:01.009387 14839 raft_consensus.cc:482] T 00000000000000000000000000000000 P be6ac173201549288841721ae3b4695a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:27:01.009505 14839 raft_consensus.cc:3037] T 00000000000000000000000000000000 P be6ac173201549288841721ae3b4695a [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:27:01.010462 14839 raft_consensus.cc:504] T 00000000000000000000000000000000 P be6ac173201549288841721ae3b4695a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be6ac173201549288841721ae3b4695a" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 36503 } }
I20260808 22:27:01.010864 14839 leader_election.cc:302] T 00000000000000000000000000000000 P be6ac173201549288841721ae3b4695a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: be6ac173201549288841721ae3b4695a; no voters:
I20260808 22:27:01.011196 14839 leader_election.cc:288] T 00000000000000000000000000000000 P be6ac173201549288841721ae3b4695a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:27:01.011324 14844 raft_consensus.cc:2781] T 00000000000000000000000000000000 P be6ac173201549288841721ae3b4695a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:27:01.011580 14844 raft_consensus.cc:686] T 00000000000000000000000000000000 P be6ac173201549288841721ae3b4695a [term 1 LEADER]: Becoming Leader. State: Replica: be6ac173201549288841721ae3b4695a, State: Running, Role: LEADER
I20260808 22:27:01.011916 14844 consensus_queue.cc:237] T 00000000000000000000000000000000 P be6ac173201549288841721ae3b4695a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be6ac173201549288841721ae3b4695a" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 36503 } }
I20260808 22:27:01.012463 14839 sys_catalog.cc:564] T 00000000000000000000000000000000 P be6ac173201549288841721ae3b4695a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:27:01.013589 14847 sys_catalog.cc:455] T 00000000000000000000000000000000 P be6ac173201549288841721ae3b4695a [sys.catalog]: SysCatalogTable state changed. Reason: New leader be6ac173201549288841721ae3b4695a. Latest consensus state: current_term: 1 leader_uuid: "be6ac173201549288841721ae3b4695a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be6ac173201549288841721ae3b4695a" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 36503 } } }
I20260808 22:27:01.013737 14847 sys_catalog.cc:458] T 00000000000000000000000000000000 P be6ac173201549288841721ae3b4695a [sys.catalog]: This master's current role is: LEADER
I20260808 22:27:01.013938 14845 sys_catalog.cc:455] T 00000000000000000000000000000000 P be6ac173201549288841721ae3b4695a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "be6ac173201549288841721ae3b4695a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "be6ac173201549288841721ae3b4695a" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 36503 } } }
I20260808 22:27:01.014092 14845 sys_catalog.cc:458] T 00000000000000000000000000000000 P be6ac173201549288841721ae3b4695a [sys.catalog]: This master's current role is: LEADER
I20260808 22:27:01.014153 14858 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:27:01.017396 14858 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:27:01.022742 14858 catalog_manager.cc:1347] Generated new cluster ID: 1f0e10cf38a847cfb070a326f16bcaaa
I20260808 22:27:01.022810 14858 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:27:01.033960 14858 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:27:01.034844 14858 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:27:01.041983 14858 catalog_manager.cc:5868] T 00000000000000000000000000000000 P be6ac173201549288841721ae3b4695a: Generated new TSK 0
I20260808 22:27:01.042652 14858 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:27:01.142938 14842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:01.053860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46394 (local address 127.5.11.254:36503)
0808 22:27:01.054002 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:01.054007 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:01.054200 (+ 193us) server_negotiation.cc:407] Connection header received
0808 22:27:01.054491 (+ 291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:01.054494 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:01.054543 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:01.054639 (+ 96us) server_negotiation.cc:226] Negotiated authn=JWT
0808 22:27:01.054915 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:01.055446 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:01.056715 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:01.056957 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:01.057044 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:01.057376 (+ 332us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0808 22:27:01.142533 (+ 85157us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 22:27:01.142811 (+ 278us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46394: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":60}
W20260808 22:27:01.143056 14842 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46394: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260808 22:27:01.143304 14888 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:01.053796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:36503 (local address 127.0.0.1:46394)
0808 22:27:01.054107 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:01.054122 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:01.054234 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:01.054676 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:01.054679 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:01.054688 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0808 22:27:01.054806 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:01.054813 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:01.055567 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:01.055571 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:01.056585 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:01.056596 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:01.057082 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:01.057086 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:01.057182 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:01.057213 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0808 22:27:01.142813 (+ 85600us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0808 22:27:01.143052 (+ 239us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.5.11.254:36503: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":117,"threads_started":1}
I20260808 22:27:01.144120 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 14726
2026-08-08T22:27:01Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (368 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (126 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (127 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:27:01 dist-test-slave-7f01 krb5kdc[14913](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:27:01 dist-test-slave-7f01 krb5kdc[14913](info): set up 2 sockets
Aug 08 22:27:01 dist-test-slave-7f01 krb5kdc[14913](info): commencing operation
krb5kdc: starting...
W20260808 22:27:02.659471 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.230s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:27:02 dist-test-slave-7f01 krb5kdc[14913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:27:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:27:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.11.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
I20260808 22:27:02.786559 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:43851
--webserver_interface=127.5.11.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:33879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:43851
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:27:02.896884 14936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:02.897185 14936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:02.897274 14936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:02.901510 14936 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:27:02.901611 14936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:02.901654 14936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:02.901713 14936 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:27:02.901754 14936 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:27:02.907111 14936 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:33879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:43851
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:43851
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:02.908159 14936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:02.909155 14936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:02.917449 14944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:02.917466 14942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:02.917469 14948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:02.917666 14936 server_base.cc:1034] running on GCE node
I20260808 22:27:02.918223 14936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:02.919327 14936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:02.920485 14936 hybrid_clock.cc:648] HybridClock initialized: now 1786228022920461 us; error 35 us; skew 500 ppm
Aug 08 22:27:02 dist-test-slave-7f01 krb5kdc[14913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228022, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:27:02.923365 14936 init.cc:377] Logged in from keytab as kudu/127.5.11.254@KRBTEST.COM (short username kudu)
I20260808 22:27:02.924270 14936 webserver.cc:460] Webserver started at http://127.5.11.254:45889/ using document root <none> and password file <none>
I20260808 22:27:02.924821 14936 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:02.924882 14936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:02.925114 14936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:27:02.927098 14936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "16d097c5ab70415aac4641e3fc7edbbe"
format_stamp: "Formatted at 2026-08-08 22:27:02 on dist-test-slave-7f01"
server_key: "04e7407e8f6987a8bb77f37c70804de5"
server_key_iv: "33ae0f6ab8b419d142db3158491feec7"
server_key_version: "encryptionkey@0"
I20260808 22:27:02.927661 14936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "16d097c5ab70415aac4641e3fc7edbbe"
format_stamp: "Formatted at 2026-08-08 22:27:02 on dist-test-slave-7f01"
server_key: "04e7407e8f6987a8bb77f37c70804de5"
server_key_iv: "33ae0f6ab8b419d142db3158491feec7"
server_key_version: "encryptionkey@0"
I20260808 22:27:02.931236 14936 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:27:02.933610 14956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:02.934805 14936 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 22:27:02.934945 14936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "16d097c5ab70415aac4641e3fc7edbbe"
format_stamp: "Formatted at 2026-08-08 22:27:02 on dist-test-slave-7f01"
server_key: "04e7407e8f6987a8bb77f37c70804de5"
server_key_iv: "33ae0f6ab8b419d142db3158491feec7"
server_key_version: "encryptionkey@0"
I20260808 22:27:02.935062 14936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:02.959182 14936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:02.961944 14936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:02.962150 14936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:02.969489 14936 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:43851
I20260808 22:27:02.969503 15032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:43851 every 8 connection(s)
I20260808 22:27:02.970432 14936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:27:02.973206 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 14936
I20260808 22:27:02.973356 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:27:02.973466 15033 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:27:02.973685 5167 external_mini_cluster.cc:1428] Setting key 2ecd6a54a543ad82915dd9565aaa67cf
I20260808 22:27:02.981091 15033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16d097c5ab70415aac4641e3fc7edbbe: Bootstrap starting.
Aug 08 22:27:02 dist-test-slave-7f01 krb5kdc[14913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:27:02.983767 15033 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16d097c5ab70415aac4641e3fc7edbbe: Neither blocks nor log segments found. Creating new log.
I20260808 22:27:02.984704 15033 log.cc:824] T 00000000000000000000000000000000 P 16d097c5ab70415aac4641e3fc7edbbe: Log is configured to *not* fsync() on all Append() calls
I20260808 22:27:02.986685 15033 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16d097c5ab70415aac4641e3fc7edbbe: No bootstrap required, opened a new log
I20260808 22:27:02.988590 15036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:02.975040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44942 (local address 127.5.11.254:43851)
0808 22:27:02.975459 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:02.975479 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:02.975533 (+ 54us) server_negotiation.cc:407] Connection header received
0808 22:27:02.976428 (+ 895us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:02.976454 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:02.976902 (+ 448us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:02.977287 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:02.977903 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:02.979363 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:02.980277 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:02.980958 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:02.981133 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:02.983936 (+ 2803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:02.983945 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:02.983965 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:02.984010 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:02.985696 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:02.986161 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:02.986166 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:02.986173 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:02.986299 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:02.986599 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:02.986602 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:02.986604 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:02.987068 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:02.987269 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:02.987578 (+ 309us) server_negotiation.cc:299] Negotiation successful
0808 22:27:02.987891 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":111,"threads_started":1}
I20260808 22:27:02.990312 15033 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16d097c5ab70415aac4641e3fc7edbbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16d097c5ab70415aac4641e3fc7edbbe" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 43851 } }
I20260808 22:27:02.990563 15033 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16d097c5ab70415aac4641e3fc7edbbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:27:02.990630 15033 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16d097c5ab70415aac4641e3fc7edbbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16d097c5ab70415aac4641e3fc7edbbe, State: Initialized, Role: FOLLOWER
I20260808 22:27:02.991093 15033 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16d097c5ab70415aac4641e3fc7edbbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16d097c5ab70415aac4641e3fc7edbbe" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 43851 } }
I20260808 22:27:02.991250 15033 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16d097c5ab70415aac4641e3fc7edbbe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:27:02.991338 15033 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16d097c5ab70415aac4641e3fc7edbbe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:27:02.991463 15033 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16d097c5ab70415aac4641e3fc7edbbe [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:27:02.992395 15033 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16d097c5ab70415aac4641e3fc7edbbe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16d097c5ab70415aac4641e3fc7edbbe" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 43851 } }
I20260808 22:27:02.992740 15033 leader_election.cc:302] T 00000000000000000000000000000000 P 16d097c5ab70415aac4641e3fc7edbbe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16d097c5ab70415aac4641e3fc7edbbe; no voters:
I20260808 22:27:02.993074 15033 leader_election.cc:288] T 00000000000000000000000000000000 P 16d097c5ab70415aac4641e3fc7edbbe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:27:02.993199 15041 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16d097c5ab70415aac4641e3fc7edbbe [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:27:02.993468 15041 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16d097c5ab70415aac4641e3fc7edbbe [term 1 LEADER]: Becoming Leader. State: Replica: 16d097c5ab70415aac4641e3fc7edbbe, State: Running, Role: LEADER
I20260808 22:27:02.993763 15041 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16d097c5ab70415aac4641e3fc7edbbe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16d097c5ab70415aac4641e3fc7edbbe" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 43851 } }
I20260808 22:27:02.994300 15033 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16d097c5ab70415aac4641e3fc7edbbe [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:27:02.995525 15044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16d097c5ab70415aac4641e3fc7edbbe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16d097c5ab70415aac4641e3fc7edbbe. Latest consensus state: current_term: 1 leader_uuid: "16d097c5ab70415aac4641e3fc7edbbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16d097c5ab70415aac4641e3fc7edbbe" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 43851 } } }
I20260808 22:27:02.995661 15044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16d097c5ab70415aac4641e3fc7edbbe [sys.catalog]: This master's current role is: LEADER
I20260808 22:27:02.996116 15053 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:27:02.995532 15043 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16d097c5ab70415aac4641e3fc7edbbe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "16d097c5ab70415aac4641e3fc7edbbe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16d097c5ab70415aac4641e3fc7edbbe" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 43851 } } }
I20260808 22:27:02.996742 15043 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16d097c5ab70415aac4641e3fc7edbbe [sys.catalog]: This master's current role is: LEADER
I20260808 22:27:02.998965 15053 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:27:03.004487 15053 catalog_manager.cc:1347] Generated new cluster ID: 1541e2a431ba4748832cac07e7ea4fb5
I20260808 22:27:03.004563 15053 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:27:03.015388 15053 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:27:03.016299 15053 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:27:03.025279 15053 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 16d097c5ab70415aac4641e3fc7edbbe: Generated new TSK 0
I20260808 22:27:03.025888 15053 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.11.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab.
Entry for principal kudu/127.5.11.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab.
I20260808 22:27:03.059185 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.193:0
--local_ip_for_outbound_sockets=127.5.11.193
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:43851
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:33879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:27:03.173024 15069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:03.173255 15069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:03.173295 15069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:03.177320 15069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:03.177369 15069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:03.177424 15069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.193
I20260808 22:27:03.182364 15069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:33879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:43851
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.193
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:03.183297 15069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:03.184362 15069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:03.193584 15081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:03.193585 15080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:03.193616 15083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:03.193876 15069 server_base.cc:1034] running on GCE node
I20260808 22:27:03.194229 15069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:03.194918 15069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:03.196120 15069 hybrid_clock.cc:648] HybridClock initialized: now 1786228023196100 us; error 30 us; skew 500 ppm
Aug 08 22:27:03 dist-test-slave-7f01 krb5kdc[14913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228023, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:27:03.199040 15069 init.cc:377] Logged in from keytab as kudu/127.5.11.193@KRBTEST.COM (short username kudu)
I20260808 22:27:03.199922 15069 webserver.cc:460] Webserver started at http://127.5.11.193:39391/ using document root <none> and password file <none>
I20260808 22:27:03.200415 15069 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:03.200472 15069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:03.200629 15069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:27:03.202492 15069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-0/data/instance:
uuid: "eefb37fff52948938829387119de67f7"
format_stamp: "Formatted at 2026-08-08 22:27:03 on dist-test-slave-7f01"
server_key: "d8baa00119c401c242b4ba9a90c12376"
server_key_iv: "ba672e0ed8955438532a67a2a1a83617"
server_key_version: "encryptionkey@0"
I20260808 22:27:03.203032 15069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance:
uuid: "eefb37fff52948938829387119de67f7"
format_stamp: "Formatted at 2026-08-08 22:27:03 on dist-test-slave-7f01"
server_key: "d8baa00119c401c242b4ba9a90c12376"
server_key_iv: "ba672e0ed8955438532a67a2a1a83617"
server_key_version: "encryptionkey@0"
I20260808 22:27:03.206844 15069 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 22:27:03.209299 15095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:03.210492 15069 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:27:03.210634 15069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-0/wal
uuid: "eefb37fff52948938829387119de67f7"
format_stamp: "Formatted at 2026-08-08 22:27:03 on dist-test-slave-7f01"
server_key: "d8baa00119c401c242b4ba9a90c12376"
server_key_iv: "ba672e0ed8955438532a67a2a1a83617"
server_key_version: "encryptionkey@0"
I20260808 22:27:03.210745 15069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:03.237857 15069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:03.241302 15069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:03.241509 15069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:03.242125 15069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:27:03.242988 15069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:27:03.243072 15069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:03.243146 15069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:27:03.243192 15069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:03.252151 15069 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.193:39687
I20260808 22:27:03.252175 15282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.193:39687 every 8 connection(s)
I20260808 22:27:03.253001 15069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
I20260808 22:27:03.255896 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 15069
I20260808 22:27:03.256032 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance
I20260808 22:27:03.256307 5167 external_mini_cluster.cc:1428] Setting key f2908a2b33ee2be8689e90b0baeb095c
Aug 08 22:27:03 dist-test-slave-7f01 krb5kdc[14913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228023, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.11.194@KRBTEST.COM; defaulting to no policy
I20260808 22:27:03.268956 15036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:03.254831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:59255 (local address 127.5.11.254:43851)
0808 22:27:03.254969 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:03.254973 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:03.255813 (+ 840us) server_negotiation.cc:407] Connection header received
0808 22:27:03.257470 (+ 1657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:03.257472 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:03.257519 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:03.257624 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:03.259343 (+ 1719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:03.260217 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:03.261986 (+ 1769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:03.262210 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:03.262289 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:03.265226 (+ 2937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:03.265230 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:03.265232 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:03.265255 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:03.266783 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:03.267316 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:03.267320 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:03.267322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:03.267412 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:03.267782 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:03.267789 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:03.267791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:03.268035 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:03.268147 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:03.268651 (+ 504us) server_negotiation.cc:299] Negotiation successful
0808 22:27:03.268784 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 22:27:03.269850 15285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:03.255189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43851 (local address 127.5.11.193:59255)
0808 22:27:03.255647 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:03.255689 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:03.257257 (+ 1568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:03.257829 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:03.257838 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:03.258408 (+ 570us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:03.259183 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:03.259200 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:03.260367 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:03.260374 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:03.261857 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:03.261872 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:03.262366 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:03.262372 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:03.262608 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:03.263214 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:03.263239 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:03.265035 (+ 1796us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:03.266908 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:03.266914 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:03.266929 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:03.267203 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:03.267511 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:03.267514 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:03.267518 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:03.267666 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:03.268145 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:03.268152 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:03.268400 (+ 248us) client_negotiation.cc:769] Sending connection context
0808 22:27:03.268594 (+ 194us) client_negotiation.cc:240] Negotiation successful
0808 22:27:03.268962 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":107,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.194@KRBTEST.COM" created.
I20260808 22:27:03.271205 15283 heartbeater.cc:347] Connected to a master server at 127.5.11.254:43851
I20260808 22:27:03.271478 15283 heartbeater.cc:464] Registering TS with master...
I20260808 22:27:03.272125 15283 heartbeater.cc:511] Master 127.5.11.254:43851 requested a full tablet report, sending...
I20260808 22:27:03.273799 14983 ts_manager.cc:194] Registered new tserver with Master: eefb37fff52948938829387119de67f7 (127.5.11.193:39687)
I20260808 22:27:03.275049 14983 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.193@KRBTEST.COM'} at 127.5.11.193:59255
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab.
Entry for principal kudu/127.5.11.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab.
I20260808 22:27:03.286940 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.194:0
--local_ip_for_outbound_sockets=127.5.11.194
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:43851
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:33879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:27:03.399973 15290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:03.400347 15290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:03.400425 15290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:03.407867 15290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:03.407929 15290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:03.407987 15290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.194
I20260808 22:27:03.413198 15290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:33879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:43851
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.194
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:03.414192 15290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:03.415227 15290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 22:27:03.424912 15290 server_base.cc:1034] running on GCE node
W20260808 22:27:03.424881 15298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:03.424867 15301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:03.424867 15303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:03.425671 15290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:03.426319 15290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:03.427515 15290 hybrid_clock.cc:648] HybridClock initialized: now 1786228023427481 us; error 43 us; skew 500 ppm
Aug 08 22:27:03 dist-test-slave-7f01 krb5kdc[14913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228023, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:27:03.430308 15290 init.cc:377] Logged in from keytab as kudu/127.5.11.194@KRBTEST.COM (short username kudu)
I20260808 22:27:03.431162 15290 webserver.cc:460] Webserver started at http://127.5.11.194:32785/ using document root <none> and password file <none>
I20260808 22:27:03.431668 15290 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:03.431728 15290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:03.431983 15290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:27:03.434000 15290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-1/data/instance:
uuid: "1e0d4a57a4674049b990b4777e0c0a05"
format_stamp: "Formatted at 2026-08-08 22:27:03 on dist-test-slave-7f01"
server_key: "bc2dba8d84c7957d94a6cb59c910ea41"
server_key_iv: "2078b3c7a345a1b3bd5913961c321785"
server_key_version: "encryptionkey@0"
I20260808 22:27:03.434523 15290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance:
uuid: "1e0d4a57a4674049b990b4777e0c0a05"
format_stamp: "Formatted at 2026-08-08 22:27:03 on dist-test-slave-7f01"
server_key: "bc2dba8d84c7957d94a6cb59c910ea41"
server_key_iv: "2078b3c7a345a1b3bd5913961c321785"
server_key_version: "encryptionkey@0"
I20260808 22:27:03.438257 15290 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260808 22:27:03.440768 15315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:03.441932 15290 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 22:27:03.442036 15290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-1/wal
uuid: "1e0d4a57a4674049b990b4777e0c0a05"
format_stamp: "Formatted at 2026-08-08 22:27:03 on dist-test-slave-7f01"
server_key: "bc2dba8d84c7957d94a6cb59c910ea41"
server_key_iv: "2078b3c7a345a1b3bd5913961c321785"
server_key_version: "encryptionkey@0"
I20260808 22:27:03.442116 15290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:03.469864 15290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:03.472553 15290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:03.472703 15290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:03.473223 15290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:27:03.473971 15290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:27:03.474032 15290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:03.474071 15290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:27:03.474092 15290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:03.482786 15290 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.194:44699
I20260808 22:27:03.482816 15489 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.194:44699 every 8 connection(s)
I20260808 22:27:03.483654 15290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
Aug 08 22:27:03 dist-test-slave-7f01 krb5kdc[14913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228023, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:27:03.493445 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 15290
I20260808 22:27:03.493532 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance
I20260808 22:27:03.493819 5167 external_mini_cluster.cc:1428] Setting key 960790a7aeedbf57be8ce173e33ac06b
I20260808 22:27:03.497452 15036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:03.485593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:48457 (local address 127.5.11.254:43851)
0808 22:27:03.485737 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:03.485741 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:03.486586 (+ 845us) server_negotiation.cc:407] Connection header received
0808 22:27:03.488007 (+ 1421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:03.488011 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:03.488063 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:03.488130 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:03.489232 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:03.489755 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:03.490819 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:03.491035 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:03.491110 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:03.493409 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:03.493413 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:03.493417 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:03.493447 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:03.495208 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:03.495859 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:03.495862 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:03.495864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:03.495951 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:03.496376 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:03.496379 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:03.496381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:03.496570 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:03.496681 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:03.497161 (+ 480us) server_negotiation.cc:299] Negotiation successful
0808 22:27:03.497284 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 22:27:03.498235 15496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:03.485991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43851 (local address 127.5.11.194:48457)
0808 22:27:03.486437 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:03.486477 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:03.487707 (+ 1230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:03.488309 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:03.488315 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:03.488650 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:03.489102 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:03.489115 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:03.489860 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:03.489862 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:03.490713 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:03.490722 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:03.491160 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:03.491162 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:03.491297 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:03.491828 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:03.491844 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:03.493243 (+ 1399us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:03.495343 (+ 2100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:03.495348 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:03.495362 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:03.495707 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:03.496053 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:03.496056 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:03.496058 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:03.496259 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:03.496680 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:03.496687 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:03.496916 (+ 229us) client_negotiation.cc:769] Sending connection context
0808 22:27:03.497100 (+ 184us) client_negotiation.cc:240] Negotiation successful
0808 22:27:03.497391 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":108,"threads_started":1}
I20260808 22:27:03.499302 15493 heartbeater.cc:347] Connected to a master server at 127.5.11.254:43851
I20260808 22:27:03.499539 15493 heartbeater.cc:464] Registering TS with master...
I20260808 22:27:03.500121 15493 heartbeater.cc:511] Master 127.5.11.254:43851 requested a full tablet report, sending...
I20260808 22:27:03.501255 14983 ts_manager.cc:194] Registered new tserver with Master: 1e0d4a57a4674049b990b4777e0c0a05 (127.5.11.194:44699)
I20260808 22:27:03.501811 14983 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.194@KRBTEST.COM'} at 127.5.11.194:48457
WARNING: no policy specified for kudu/127.5.11.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab.
Entry for principal kudu/127.5.11.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab.
I20260808 22:27:03.523190 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.195:0
--local_ip_for_outbound_sockets=127.5.11.195
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:43851
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:33879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:27:03.633386 15499 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:03.633647 15499 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:03.633744 15499 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:03.638327 15499 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:03.638403 15499 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:03.638527 15499 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.195
I20260808 22:27:03.643541 15499 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:33879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:43851
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15499
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.195
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:03.644466 15499 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:03.645416 15499 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:03.654552 15516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:03.654560 15510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:03.654657 15512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:03.654898 15499 server_base.cc:1034] running on GCE node
I20260808 22:27:03.655282 15499 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:03.655936 15499 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:03.657096 15499 hybrid_clock.cc:648] HybridClock initialized: now 1786228023657078 us; error 28 us; skew 500 ppm
Aug 08 22:27:03 dist-test-slave-7f01 krb5kdc[14913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228023, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:27:03.659921 15499 init.cc:377] Logged in from keytab as kudu/127.5.11.195@KRBTEST.COM (short username kudu)
I20260808 22:27:03.660727 15499 webserver.cc:460] Webserver started at http://127.5.11.195:37405/ using document root <none> and password file <none>
I20260808 22:27:03.661263 15499 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:03.661360 15499 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:03.661597 15499 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:27:03.663604 15499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-2/data/instance:
uuid: "e5650dd838ad438bb6c0ac78c245c775"
format_stamp: "Formatted at 2026-08-08 22:27:03 on dist-test-slave-7f01"
server_key: "ae0a3878f0e6bea987661a27dceaa838"
server_key_iv: "1bcc5102921c5ea9854bccece86aaa0b"
server_key_version: "encryptionkey@0"
I20260808 22:27:03.664294 15499 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance:
uuid: "e5650dd838ad438bb6c0ac78c245c775"
format_stamp: "Formatted at 2026-08-08 22:27:03 on dist-test-slave-7f01"
server_key: "ae0a3878f0e6bea987661a27dceaa838"
server_key_iv: "1bcc5102921c5ea9854bccece86aaa0b"
server_key_version: "encryptionkey@0"
I20260808 22:27:03.668182 15499 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 22:27:03.670526 15530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:03.671901 15499 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:27:03.672025 15499 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-2/wal
uuid: "e5650dd838ad438bb6c0ac78c245c775"
format_stamp: "Formatted at 2026-08-08 22:27:03 on dist-test-slave-7f01"
server_key: "ae0a3878f0e6bea987661a27dceaa838"
server_key_iv: "1bcc5102921c5ea9854bccece86aaa0b"
server_key_version: "encryptionkey@0"
I20260808 22:27:03.672127 15499 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:03.682408 15499 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:03.685132 15499 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:03.685369 15499 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:03.685913 15499 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:27:03.686684 15499 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:27:03.686733 15499 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:03.686790 15499 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:27:03.686817 15499 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:03.695837 15499 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.195:44409
I20260808 22:27:03.695878 15722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.195:44409 every 8 connection(s)
I20260808 22:27:03.696727 15499 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
I20260808 22:27:03.698879 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 15499
I20260808 22:27:03.699019 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance
I20260808 22:27:03.699311 5167 external_mini_cluster.cc:1428] Setting key 84201252dacc9483ad4c300df6c08212
Aug 08 22:27:03 dist-test-slave-7f01 krb5kdc[14913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228023, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:27:03.710047 15036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:03.698450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:57995 (local address 127.5.11.254:43851)
0808 22:27:03.698584 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:03.698588 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:03.699546 (+ 958us) server_negotiation.cc:407] Connection header received
0808 22:27:03.701175 (+ 1629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:03.701178 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:03.701236 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:03.701321 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:03.702406 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:03.702917 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:03.704123 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:03.704369 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:03.704485 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:03.706764 (+ 2279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:03.706777 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:03.706779 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:03.706804 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:03.708269 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:03.708713 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:03.708716 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:03.708717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:03.708794 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:03.709132 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:03.709134 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:03.709136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:03.709302 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:03.709378 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:03.709821 (+ 443us) server_negotiation.cc:299] Negotiation successful
0808 22:27:03.709921 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 22:27:03.710822 15728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:03.698817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43851 (local address 127.5.11.195:57995)
0808 22:27:03.699386 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:03.699436 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:03.700974 (+ 1538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:03.701514 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:03.701520 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:03.701845 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:03.702283 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:03.702295 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:03.703045 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:03.703048 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:03.703977 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:03.703988 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:03.704521 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:03.704526 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:03.704695 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:03.705157 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:03.705173 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:03.706614 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:03.708367 (+ 1753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:03.708372 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:03.708382 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:03.708628 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:03.708888 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:03.708895 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:03.708900 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:03.709049 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:03.709391 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:03.709396 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:03.709613 (+ 217us) client_negotiation.cc:769] Sending connection context
0808 22:27:03.709780 (+ 167us) client_negotiation.cc:240] Negotiation successful
0808 22:27:03.710016 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":147,"threads_started":1}
I20260808 22:27:03.712033 15723 heartbeater.cc:347] Connected to a master server at 127.5.11.254:43851
I20260808 22:27:03.712289 15723 heartbeater.cc:464] Registering TS with master...
I20260808 22:27:03.712877 15723 heartbeater.cc:511] Master 127.5.11.254:43851 requested a full tablet report, sending...
I20260808 22:27:03.713922 14983 ts_manager.cc:194] Registered new tserver with Master: e5650dd838ad438bb6c0ac78c245c775 (127.5.11.195:44409)
I20260808 22:27:03.714475 14983 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.195@KRBTEST.COM'} at 127.5.11.195:57995
I20260808 22:27:03.725059 5167 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/krb5kdc/test-admin.keytab.
Aug 08 22:27:03 dist-test-slave-7f01 krb5kdc[14913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:27:03.741755 5167 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260808 22:27:03.744434 15734 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:03.743028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49760 (local address 127.5.11.193:39687)
0808 22:27:03.743328 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:03.743343 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:03.743366 (+ 23us) server_negotiation.cc:407] Connection header received
0808 22:27:03.743413 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:03.743417 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:03.743605 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:03.743723 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:03.744256 (+ 533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49760: BlockingRecv error: recv got EOF from 127.0.0.1:49760 (error 108)
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":110,"threads_started":1}
W20260808 22:27:03.744473 5167 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/krb5kdc/krb5cc' of type FILE
Aug 08 22:27:03 dist-test-slave-7f01 krb5kdc[14913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:27:03.745771 5167 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 22:27:03 dist-test-slave-7f01 krb5kdc[14913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:27:03.746730 5167 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 22:27:03 dist-test-slave-7f01 krb5kdc[14913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.193@KRBTEST.COM
I20260808 22:27:03.756067 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:03.747585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49766 (local address 127.5.11.193:39687)
0808 22:27:03.747714 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:03.747719 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:03.747842 (+ 123us) server_negotiation.cc:407] Connection header received
0808 22:27:03.747985 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:03.747990 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:03.748034 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:03.748121 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:03.748640 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:03.749401 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:03.750232 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:03.750611 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:03.750729 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:03.752810 (+ 2081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:03.752818 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:03.752829 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:03.752851 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:03.754526 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:03.754883 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:03.754887 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:03.754889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:03.754966 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:03.755190 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:03.755192 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:03.755193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:03.755407 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:03.755557 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:03.755700 (+ 143us) server_negotiation.cc:299] Negotiation successful
0808 22:27:03.755909 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260808 22:27:03.759666 15734 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:03.758661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49768 (local address 127.5.11.193:39687)
0808 22:27:03.758781 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:03.758784 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:03.758885 (+ 101us) server_negotiation.cc:407] Connection header received
0808 22:27:03.759030 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:03.759033 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:03.759075 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:03.759158 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:03.759554 (+ 396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49768: BlockingRecv error: recv got EOF from 127.0.0.1:49768 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260808 22:27:03.759860 5167 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/krb5kdc/krb5cc' of type FILE
Aug 08 22:27:03 dist-test-slave-7f01 krb5kdc[14913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:27:03.760793 5167 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 22:27:03 dist-test-slave-7f01 krb5kdc[14913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:27:03.761693 5167 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 22:27:03 dist-test-slave-7f01 krb5kdc[14913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.193@KRBTEST.COM
I20260808 22:27:03.770000 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:03.762423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49780 (local address 127.5.11.193:39687)
0808 22:27:03.762548 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:03.762552 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:03.762662 (+ 110us) server_negotiation.cc:407] Connection header received
0808 22:27:03.762803 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:03.762806 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:03.762845 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:03.762921 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:03.763391 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:03.763981 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:03.764752 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:03.764973 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:03.765084 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:03.767119 (+ 2035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:03.767124 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:03.767132 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:03.767153 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:03.768414 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:03.768835 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:03.768839 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:03.768841 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:03.768922 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:03.769222 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:03.769226 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:03.769228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:03.769386 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:03.769517 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:03.769759 (+ 242us) server_negotiation.cc:299] Negotiation successful
0808 22:27:03.769868 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260808 22:27:03.772627 15734 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:03.771551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49786 (local address 127.5.11.193:39687)
0808 22:27:03.771726 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:03.771730 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:03.771881 (+ 151us) server_negotiation.cc:407] Connection header received
0808 22:27:03.772021 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:03.772024 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:03.772067 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:03.772146 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:03.772520 (+ 374us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49786: BlockingRecv error: recv got EOF from 127.0.0.1:49786 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260808 22:27:03.772990 5167 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786227991988852-5167-0/krb5kdc/krb5cc)
Aug 08 22:27:03 dist-test-slave-7f01 krb5kdc[14913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:27:03.773923 5167 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 08 22:27:03 dist-test-slave-7f01 krb5kdc[14913](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:27:03.774775 5167 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 08 22:27:03 dist-test-slave-7f01 krb5kdc[14913](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228023, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.193@KRBTEST.COM
I20260808 22:27:03.782806 15734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:03.775531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49792 (local address 127.5.11.193:39687)
0808 22:27:03.775690 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:03.775693 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:03.775782 (+ 89us) server_negotiation.cc:407] Connection header received
0808 22:27:03.775883 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:03.775885 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:03.775917 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:03.775981 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:03.776442 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:03.777000 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:03.777696 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:03.777918 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:03.777994 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:03.780031 (+ 2037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:03.780034 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:03.780036 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:03.780057 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:03.781448 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:03.781817 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:03.781819 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:03.781820 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:03.781894 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:03.782118 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:03.782120 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:03.782121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:03.782260 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:03.782353 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:03.782510 (+ 157us) server_negotiation.cc:299] Negotiation successful
0808 22:27:03.782588 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260808 22:27:03.783699 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 15069
I20260808 22:27:03.800199 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 15290
I20260808 22:27:03.827214 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 15499
I20260808 22:27:03.845395 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 14936
2026-08-08T22:27:03Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2441 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:27:03 dist-test-slave-7f01 krb5kdc[15754](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:27:03 dist-test-slave-7f01 krb5kdc[15754](info): set up 2 sockets
Aug 08 22:27:03 dist-test-slave-7f01 krb5kdc[15754](info): commencing operation
krb5kdc: starting...
W20260808 22:27:05.099089 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.228s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:27:05 dist-test-slave-7f01 krb5kdc[15754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:27:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:27:05Z Disabled control of system clock
WARNING: no policy specified for oryx/127.5.11.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.11.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.11.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.254.keytab.
Entry for principal oryx/127.5.11.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.254.keytab.
I20260808 22:27:05.227372 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:39325
--webserver_interface=127.5.11.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.254.keytab
--principal=oryx/127.5.11.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:46417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:39325
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:27:05.340368 15776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:05.340637 15776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:05.340675 15776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:05.344571 15776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:27:05.344620 15776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:05.344640 15776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:05.344656 15776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:27:05.344671 15776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:27:05.349663 15776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:46417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:39325
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.254.keytab
--principal=oryx/127.5.11.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:39325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:05.350586 15776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:05.351521 15776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 22:27:05.360008 15776 server_base.cc:1034] running on GCE node
W20260808 22:27:05.360033 15782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:05.359997 15784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:05.360013 15787 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:05.360781 15776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:05.361819 15776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:05.363025 15776 hybrid_clock.cc:648] HybridClock initialized: now 1786228025363018 us; error 44 us; skew 500 ppm
Aug 08 22:27:05 dist-test-slave-7f01 krb5kdc[15754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228025, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:27:05.365722 15776 init.cc:377] Logged in from keytab as oryx/127.5.11.254@KRBTEST.COM (short username oryx)
I20260808 22:27:05.366616 15776 webserver.cc:460] Webserver started at http://127.5.11.254:45855/ using document root <none> and password file <none>
I20260808 22:27:05.367187 15776 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:05.367283 15776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:05.367530 15776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:27:05.369603 15776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "b62887ed9ca74e17a55fd6afeadb4cf6"
format_stamp: "Formatted at 2026-08-08 22:27:05 on dist-test-slave-7f01"
server_key: "ee08b154f0ef31a632d47e5a63049865"
server_key_iv: "2ccdb0a78ec9e7a36b7d4254574349c4"
server_key_version: "encryptionkey@0"
I20260808 22:27:05.370204 15776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "b62887ed9ca74e17a55fd6afeadb4cf6"
format_stamp: "Formatted at 2026-08-08 22:27:05 on dist-test-slave-7f01"
server_key: "ee08b154f0ef31a632d47e5a63049865"
server_key_iv: "2ccdb0a78ec9e7a36b7d4254574349c4"
server_key_version: "encryptionkey@0"
I20260808 22:27:05.374049 15776 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 22:27:05.376385 15798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:05.377575 15776 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:27:05.377722 15776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "b62887ed9ca74e17a55fd6afeadb4cf6"
format_stamp: "Formatted at 2026-08-08 22:27:05 on dist-test-slave-7f01"
server_key: "ee08b154f0ef31a632d47e5a63049865"
server_key_iv: "2ccdb0a78ec9e7a36b7d4254574349c4"
server_key_version: "encryptionkey@0"
I20260808 22:27:05.377836 15776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:05.395277 15776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:05.398170 15776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:05.398388 15776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:05.405771 15776 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:39325
I20260808 22:27:05.406095 15883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:39325 every 8 connection(s)
I20260808 22:27:05.406838 15776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:27:05.409832 15886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:27:05.414181 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 15776
I20260808 22:27:05.414327 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:27:05.414616 5167 external_mini_cluster.cc:1428] Setting key c4229b7edac51b8c18fe5470492eb24f
I20260808 22:27:05.416111 15886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6: Bootstrap starting.
I20260808 22:27:05.418741 15886 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6: Neither blocks nor log segments found. Creating new log.
I20260808 22:27:05.419576 15886 log.cc:824] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6: Log is configured to *not* fsync() on all Append() calls
I20260808 22:27:05.421753 15886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6: No bootstrap required, opened a new log
Aug 08 22:27:05 dist-test-slave-7f01 krb5kdc[15754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM
I20260808 22:27:05.425827 15886 raft_consensus.cc:348] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } }
I20260808 22:27:05.426046 15886 raft_consensus.cc:374] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:27:05.426107 15886 raft_consensus.cc:729] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b62887ed9ca74e17a55fd6afeadb4cf6, State: Initialized, Role: FOLLOWER
I20260808 22:27:05.426529 15886 consensus_queue.cc:260] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } }
I20260808 22:27:05.426688 15886 raft_consensus.cc:388] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:27:05.426801 15886 raft_consensus.cc:482] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:27:05.426940 15886 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:27:05.427865 15886 raft_consensus.cc:504] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } }
I20260808 22:27:05.428211 15886 leader_election.cc:302] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b62887ed9ca74e17a55fd6afeadb4cf6; no voters:
I20260808 22:27:05.428606 15886 leader_election.cc:288] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:27:05.428721 15895 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:27:05.428896 15895 raft_consensus.cc:686] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [term 1 LEADER]: Becoming Leader. State: Replica: b62887ed9ca74e17a55fd6afeadb4cf6, State: Running, Role: LEADER
I20260808 22:27:05.429214 15895 consensus_queue.cc:237] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } }
I20260808 22:27:05.429885 15892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:05.416106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60496 (local address 127.5.11.254:39325)
0808 22:27:05.416569 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:05.416586 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:05.416621 (+ 35us) server_negotiation.cc:407] Connection header received
0808 22:27:05.417250 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:05.417266 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:05.417586 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:05.417860 (+ 274us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:05.418801 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:05.419976 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:05.421110 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:05.421562 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:05.421682 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:05.424546 (+ 2864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:05.424552 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:05.424564 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:05.424591 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:05.426540 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:05.427080 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:05.427088 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:05.427098 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:05.427212 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:05.427507 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:05.427514 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:05.427516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:05.428013 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:05.428249 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:05.428488 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 22:27:05.428963 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":151,"threads_started":1}
I20260808 22:27:05.429986 15886 sys_catalog.cc:564] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:27:05.430379 15898 sys_catalog.cc:455] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b62887ed9ca74e17a55fd6afeadb4cf6. Latest consensus state: current_term: 1 leader_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } }
I20260808 22:27:05.430522 15898 sys_catalog.cc:458] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [sys.catalog]: This master's current role is: LEADER
I20260808 22:27:05.430922 15901 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:27:05.432852 15896 sys_catalog.cc:455] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } }
I20260808 22:27:05.432991 15896 sys_catalog.cc:458] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [sys.catalog]: This master's current role is: LEADER
I20260808 22:27:05.435272 15901 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:27:05.441525 15901 catalog_manager.cc:1347] Generated new cluster ID: e6cced0e5a9e49409b85ee0fb2c9be1c
I20260808 22:27:05.441604 15901 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:27:05.464800 15901 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:27:05.465761 15901 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:27:05.470392 15901 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6: Generated new TSK 0
I20260808 22:27:05.471081 15901 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.5.11.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.11.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.11.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.193.keytab.
Entry for principal oryx/127.5.11.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.193.keytab.
I20260808 22:27:05.502934 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.193:0
--local_ip_for_outbound_sockets=127.5.11.193
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:39325
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.193.keytab
--principal=oryx/127.5.11.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:46417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:27:05.617837 15929 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:05.618095 15929 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:05.618136 15929 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:05.622077 15929 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:05.622130 15929 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:05.622184 15929 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.193
I20260808 22:27:05.627175 15929 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:46417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.193.keytab
--principal=oryx/127.5.11.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:39325
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15929
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.193
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:05.628135 15929 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:05.629158 15929 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:05.638826 15945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:05.638931 15943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:05.638943 15929 server_base.cc:1034] running on GCE node
W20260808 22:27:05.638826 15948 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:05.639533 15929 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:05.640198 15929 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:05.641367 15929 hybrid_clock.cc:648] HybridClock initialized: now 1786228025641333 us; error 45 us; skew 500 ppm
Aug 08 22:27:05 dist-test-slave-7f01 krb5kdc[15754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228025, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:27:05.644151 15929 init.cc:377] Logged in from keytab as oryx/127.5.11.193@KRBTEST.COM (short username oryx)
I20260808 22:27:05.644980 15929 webserver.cc:460] Webserver started at http://127.5.11.193:39243/ using document root <none> and password file <none>
I20260808 22:27:05.645542 15929 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:05.645637 15929 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:05.645882 15929 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:27:05.647958 15929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-0/data/instance:
uuid: "f43f8e31fb01495fabaefe56425dc079"
format_stamp: "Formatted at 2026-08-08 22:27:05 on dist-test-slave-7f01"
server_key: "d567ffe22fe5ff61d01ccb999fd9a258"
server_key_iv: "2142c2d1bfdc3c55f25cf0d9399b6309"
server_key_version: "encryptionkey@0"
I20260808 22:27:05.648547 15929 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance:
uuid: "f43f8e31fb01495fabaefe56425dc079"
format_stamp: "Formatted at 2026-08-08 22:27:05 on dist-test-slave-7f01"
server_key: "d567ffe22fe5ff61d01ccb999fd9a258"
server_key_iv: "2142c2d1bfdc3c55f25cf0d9399b6309"
server_key_version: "encryptionkey@0"
I20260808 22:27:05.652580 15929 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 22:27:05.655016 15957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:05.656294 15929 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:27:05.656443 15929 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-0/wal
uuid: "f43f8e31fb01495fabaefe56425dc079"
format_stamp: "Formatted at 2026-08-08 22:27:05 on dist-test-slave-7f01"
server_key: "d567ffe22fe5ff61d01ccb999fd9a258"
server_key_iv: "2142c2d1bfdc3c55f25cf0d9399b6309"
server_key_version: "encryptionkey@0"
I20260808 22:27:05.656555 15929 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:05.665705 15929 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:05.668596 15929 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:05.668793 15929 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:05.669359 15929 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:27:05.670233 15929 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:27:05.670301 15929 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:05.670361 15929 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:27:05.670413 15929 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:05.679625 15929 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.193:39203
I20260808 22:27:05.679651 16138 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.193:39203 every 8 connection(s)
I20260808 22:27:05.680518 15929 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
Aug 08 22:27:05 dist-test-slave-7f01 krb5kdc[15754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228025, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM
I20260808 22:27:05.690133 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 15929
I20260808 22:27:05.690274 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance
I20260808 22:27:05.690654 5167 external_mini_cluster.cc:1428] Setting key ff4dd5c805cfd54bfa36e1b3b5f38872
I20260808 22:27:05.694037 15892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:05.682467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:56883 (local address 127.5.11.254:39325)
0808 22:27:05.682596 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:05.682600 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:05.683584 (+ 984us) server_negotiation.cc:407] Connection header received
0808 22:27:05.684457 (+ 873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:05.684459 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:05.684508 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:05.684580 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:05.685660 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:05.686194 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:05.687335 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:05.687565 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:05.687650 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:05.690132 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:05.690136 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:05.690139 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:05.690171 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:05.691856 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:05.692486 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:05.692491 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:05.692493 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:05.692586 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:05.692941 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:05.692945 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:05.692947 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:05.693131 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:05.693240 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:05.693737 (+ 497us) server_negotiation.cc:299] Negotiation successful
0808 22:27:05.693866 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 22:27:05.694795 16142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:05.682780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.5.11.193:56883)
0808 22:27:05.683434 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:05.683474 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:05.684270 (+ 796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:05.684740 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:05.684746 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:05.685083 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:05.685530 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:05.685543 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:05.686302 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:05.686305 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:05.687198 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:05.687207 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:05.687699 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:05.687702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:05.687910 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:05.688504 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:05.688521 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:05.689956 (+ 1435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:05.691988 (+ 2032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:05.691994 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:05.692015 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:05.692372 (+ 357us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:05.692681 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:05.692684 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:05.692686 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:05.692835 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:05.693254 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:05.693263 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:05.693503 (+ 240us) client_negotiation.cc:769] Sending connection context
0808 22:27:05.693692 (+ 189us) client_negotiation.cc:240] Negotiation successful
0808 22:27:05.694004 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":106,"threads_started":1}
I20260808 22:27:05.696094 16139 heartbeater.cc:347] Connected to a master server at 127.5.11.254:39325
I20260808 22:27:05.696337 16139 heartbeater.cc:464] Registering TS with master...
I20260808 22:27:05.696923 16139 heartbeater.cc:511] Master 127.5.11.254:39325 requested a full tablet report, sending...
I20260808 22:27:05.698591 15823 ts_manager.cc:194] Registered new tserver with Master: f43f8e31fb01495fabaefe56425dc079 (127.5.11.193:39203)
I20260808 22:27:05.699833 15823 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.11.193@KRBTEST.COM'} at 127.5.11.193:56883
WARNING: no policy specified for oryx/127.5.11.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.11.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.11.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.194.keytab.
Entry for principal oryx/127.5.11.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.194.keytab.
I20260808 22:27:05.719445 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.194:0
--local_ip_for_outbound_sockets=127.5.11.194
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:39325
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.194.keytab
--principal=oryx/127.5.11.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:46417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:27:05.832839 16149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:05.833115 16149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:05.833190 16149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:05.837278 16149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:05.837339 16149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:05.837428 16149 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.194
I20260808 22:27:05.842470 16149 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:46417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.194.keytab
--principal=oryx/127.5.11.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:39325
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.194
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:05.843505 16149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:05.844594 16149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:05.854080 16158 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:05.854111 16154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:05.854090 16155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:05.854476 16149 server_base.cc:1034] running on GCE node
I20260808 22:27:05.854930 16149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:05.855638 16149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:05.856828 16149 hybrid_clock.cc:648] HybridClock initialized: now 1786228025856797 us; error 52 us; skew 500 ppm
Aug 08 22:27:05 dist-test-slave-7f01 krb5kdc[15754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228025, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:27:05.859593 16149 init.cc:377] Logged in from keytab as oryx/127.5.11.194@KRBTEST.COM (short username oryx)
I20260808 22:27:05.860575 16149 webserver.cc:460] Webserver started at http://127.5.11.194:46407/ using document root <none> and password file <none>
I20260808 22:27:05.861194 16149 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:05.861265 16149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:05.861500 16149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:27:05.863548 16149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-1/data/instance:
uuid: "bc97c1ed9ee6470b99e8b230c5c84b61"
format_stamp: "Formatted at 2026-08-08 22:27:05 on dist-test-slave-7f01"
server_key: "5a2b0e738d9803a25b8fd64da8848658"
server_key_iv: "b4a427ff5555803c4ba3d5be2a993d1b"
server_key_version: "encryptionkey@0"
I20260808 22:27:05.864204 16149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance:
uuid: "bc97c1ed9ee6470b99e8b230c5c84b61"
format_stamp: "Formatted at 2026-08-08 22:27:05 on dist-test-slave-7f01"
server_key: "5a2b0e738d9803a25b8fd64da8848658"
server_key_iv: "b4a427ff5555803c4ba3d5be2a993d1b"
server_key_version: "encryptionkey@0"
I20260808 22:27:05.868085 16149 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 22:27:05.870436 16168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:05.871697 16149 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:27:05.871868 16149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-1/wal
uuid: "bc97c1ed9ee6470b99e8b230c5c84b61"
format_stamp: "Formatted at 2026-08-08 22:27:05 on dist-test-slave-7f01"
server_key: "5a2b0e738d9803a25b8fd64da8848658"
server_key_iv: "b4a427ff5555803c4ba3d5be2a993d1b"
server_key_version: "encryptionkey@0"
I20260808 22:27:05.871984 16149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:05.894359 16149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:05.897351 16149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:05.897559 16149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:05.898172 16149 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:27:05.899014 16149 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:27:05.899091 16149 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:05.899160 16149 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:27:05.899214 16149 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:05.908286 16149 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.194:43785
I20260808 22:27:05.908311 16344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.194:43785 every 8 connection(s)
I20260808 22:27:05.909155 16149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
I20260808 22:27:05.916625 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 16149
I20260808 22:27:05.916778 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance
I20260808 22:27:05.917098 5167 external_mini_cluster.cc:1428] Setting key 70012459a7b2298871a5fc6782aeac72
Aug 08 22:27:05 dist-test-slave-7f01 krb5kdc[15754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228025, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM
I20260808 22:27:05.922740 15892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:05.911018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:57495 (local address 127.5.11.254:39325)
0808 22:27:05.911156 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:05.911160 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:05.912249 (+ 1089us) server_negotiation.cc:407] Connection header received
0808 22:27:05.913136 (+ 887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:05.913139 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:05.913196 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:05.913310 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:05.914461 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:05.914987 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:05.916208 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:05.916456 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:05.916573 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:05.919188 (+ 2615us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:05.919191 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:05.919193 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:05.919217 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:05.920683 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:05.921203 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:05.921207 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:05.921209 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:05.921295 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:05.921645 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:05.921650 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:05.921652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:05.921836 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:05.921950 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:05.922436 (+ 486us) server_negotiation.cc:299] Negotiation successful
0808 22:27:05.922568 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 22:27:05.923487 16360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:05.911385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.5.11.194:57495)
0808 22:27:05.912106 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:05.912147 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:05.912931 (+ 784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:05.913495 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:05.913502 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:05.913846 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:05.914311 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:05.914325 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:05.915126 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:05.915134 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:05.916081 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:05.916092 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:05.916609 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:05.916613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:05.916821 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:05.917452 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:05.917467 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:05.919018 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:05.920806 (+ 1788us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:05.920812 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:05.920826 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:05.921096 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:05.921390 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:05.921395 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:05.921397 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:05.921545 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:05.921947 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:05.921953 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:05.922197 (+ 244us) client_negotiation.cc:769] Sending connection context
0808 22:27:05.922388 (+ 191us) client_negotiation.cc:240] Negotiation successful
0808 22:27:05.922695 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":535,"thread_start_us":131,"threads_started":1}
I20260808 22:27:05.924799 16345 heartbeater.cc:347] Connected to a master server at 127.5.11.254:39325
I20260808 22:27:05.925037 16345 heartbeater.cc:464] Registering TS with master...
I20260808 22:27:05.925516 16345 heartbeater.cc:511] Master 127.5.11.254:39325 requested a full tablet report, sending...
I20260808 22:27:05.926566 15823 ts_manager.cc:194] Registered new tserver with Master: bc97c1ed9ee6470b99e8b230c5c84b61 (127.5.11.194:43785)
I20260808 22:27:05.927392 15823 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.11.194@KRBTEST.COM'} at 127.5.11.194:57495
WARNING: no policy specified for oryx/127.5.11.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.11.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.11.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.195.keytab.
Entry for principal oryx/127.5.11.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.195.keytab.
I20260808 22:27:05.946197 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.195:0
--local_ip_for_outbound_sockets=127.5.11.195
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:39325
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.195.keytab
--principal=oryx/127.5.11.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:46417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:27:06.061578 16366 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:06.061820 16366 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:06.061859 16366 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:06.065757 16366 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:06.065806 16366 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:06.065861 16366 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.195
I20260808 22:27:06.070850 16366 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:46417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.195.keytab
--principal=oryx/127.5.11.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:39325
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16366
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.195
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:06.071866 16366 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:06.072808 16366 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:06.082616 16372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:06.082614 16371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:06.082623 16374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:06.082876 16366 server_base.cc:1034] running on GCE node
I20260808 22:27:06.083262 16366 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:06.083946 16366 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:06.085139 16366 hybrid_clock.cc:648] HybridClock initialized: now 1786228026085120 us; error 33 us; skew 500 ppm
Aug 08 22:27:06 dist-test-slave-7f01 krb5kdc[15754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228026, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:27:06.087893 16366 init.cc:377] Logged in from keytab as oryx/127.5.11.195@KRBTEST.COM (short username oryx)
I20260808 22:27:06.088734 16366 webserver.cc:460] Webserver started at http://127.5.11.195:38479/ using document root <none> and password file <none>
I20260808 22:27:06.089298 16366 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:06.089393 16366 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:06.089630 16366 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:27:06.091805 16366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-2/data/instance:
uuid: "feed9713203a46bfb1c3a49ba5cecd66"
format_stamp: "Formatted at 2026-08-08 22:27:06 on dist-test-slave-7f01"
server_key: "6216b51792a0f620b8c46b5910d4a7b6"
server_key_iv: "31ababbe7d430b00b69ea534ce7cad73"
server_key_version: "encryptionkey@0"
I20260808 22:27:06.092392 16366 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance:
uuid: "feed9713203a46bfb1c3a49ba5cecd66"
format_stamp: "Formatted at 2026-08-08 22:27:06 on dist-test-slave-7f01"
server_key: "6216b51792a0f620b8c46b5910d4a7b6"
server_key_iv: "31ababbe7d430b00b69ea534ce7cad73"
server_key_version: "encryptionkey@0"
I20260808 22:27:06.096086 16366 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:27:06.098450 16386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:06.099676 16366 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:27:06.099834 16366 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-2/wal
uuid: "feed9713203a46bfb1c3a49ba5cecd66"
format_stamp: "Formatted at 2026-08-08 22:27:06 on dist-test-slave-7f01"
server_key: "6216b51792a0f620b8c46b5910d4a7b6"
server_key_iv: "31ababbe7d430b00b69ea534ce7cad73"
server_key_version: "encryptionkey@0"
I20260808 22:27:06.099948 16366 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:06.109956 16366 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:06.112782 16366 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:06.112979 16366 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:06.113552 16366 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:27:06.114329 16366 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:27:06.114401 16366 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:06.114463 16366 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:27:06.114513 16366 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:06.123136 16366 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.195:45457
I20260808 22:27:06.123207 16575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.195:45457 every 8 connection(s)
I20260808 22:27:06.124056 16366 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
I20260808 22:27:06.132452 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 16366
I20260808 22:27:06.132624 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance
I20260808 22:27:06.132934 5167 external_mini_cluster.cc:1428] Setting key 483c9f3db88adc0a92ee41733afe8d9c
Aug 08 22:27:06 dist-test-slave-7f01 krb5kdc[15754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228026, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM
I20260808 22:27:06.137626 15892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.125811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:50035 (local address 127.5.11.254:39325)
0808 22:27:06.125981 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:06.125986 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:06.126980 (+ 994us) server_negotiation.cc:407] Connection header received
0808 22:27:06.127884 (+ 904us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:06.127887 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:06.127936 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:06.128015 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:06.129114 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.129627 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.130777 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.130982 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.131067 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:06.133767 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:06.133774 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:06.133781 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:06.133804 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:06.135425 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:06.136040 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:06.136045 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:06.136047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:06.136153 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:06.136511 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:06.136514 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:06.136516 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:06.136696 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:06.136805 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:06.137330 (+ 525us) server_negotiation.cc:299] Negotiation successful
0808 22:27:06.137449 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 22:27:06.138321 16579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.126162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.5.11.195:50035)
0808 22:27:06.126831 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:06.126881 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:06.127624 (+ 743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:06.128171 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:06.128178 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:06.128528 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:06.128985 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:06.128998 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.129734 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.129737 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:06.130674 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:06.130684 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.131142 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.131148 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:06.131309 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:06.131812 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:06.131828 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:06.133565 (+ 1737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:06.135559 (+ 1994us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:06.135565 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:06.135579 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:06.135932 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:06.136248 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:06.136251 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:06.136254 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:06.136398 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:06.136819 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:06.136827 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:06.137067 (+ 240us) client_negotiation.cc:769] Sending connection context
0808 22:27:06.137266 (+ 199us) client_negotiation.cc:240] Negotiation successful
0808 22:27:06.137571 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":107,"threads_started":1}
I20260808 22:27:06.139340 16577 heartbeater.cc:347] Connected to a master server at 127.5.11.254:39325
I20260808 22:27:06.139588 16577 heartbeater.cc:464] Registering TS with master...
I20260808 22:27:06.140093 16577 heartbeater.cc:511] Master 127.5.11.254:39325 requested a full tablet report, sending...
I20260808 22:27:06.141062 15823 ts_manager.cc:194] Registered new tserver with Master: feed9713203a46bfb1c3a49ba5cecd66 (127.5.11.195:45457)
I20260808 22:27:06.141606 15823 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.11.195@KRBTEST.COM'} at 127.5.11.195:50035
I20260808 22:27:06.146836 5167 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.5.11.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.11.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.11.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.253.keytab.
Entry for principal oryx/127.5.11.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.253.keytab.
I20260808 22:27:06.174743 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.253:37661
--webserver_interface=127.5.11.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.253.keytab
--principal=oryx/127.5.11.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:46417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:39325,127.5.11.253:37661
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:27:06.286753 16588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:06.287060 16588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:06.287143 16588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:06.291145 16588 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:27:06.291226 16588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:06.291263 16588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:06.291319 16588 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:27:06.291355 16588 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:27:06.296649 16588 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:46417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:39325,127.5.11.253:37661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.253.keytab
--principal=oryx/127.5.11.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.253:37661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.11.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
W20260808 22:27:06.296888 16588 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.5.11.254:39325,127.5.11.253:37661'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260808 22:27:06.297716 16588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:06.298707 16588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:06.307472 16594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:06.307457 16595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:06.307523 16588 server_base.cc:1034] running on GCE node
W20260808 22:27:06.307457 16598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:06.308257 16588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:06.309238 16588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:06.310392 16588 hybrid_clock.cc:648] HybridClock initialized: now 1786228026310356 us; error 48 us; skew 500 ppm
Aug 08 22:27:06 dist-test-slave-7f01 krb5kdc[15754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228026, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:27:06.313165 16588 init.cc:377] Logged in from keytab as oryx/127.5.11.253@KRBTEST.COM (short username oryx)
I20260808 22:27:06.314071 16588 webserver.cc:460] Webserver started at http://127.5.11.253:46349/ using document root <none> and password file <none>
I20260808 22:27:06.314620 16588 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:06.314682 16588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:06.314918 16588 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:27:06.316970 16588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-1/data/instance:
uuid: "616fc1115cca4edc9017bcb25fc8f143"
format_stamp: "Formatted at 2026-08-08 22:27:06 on dist-test-slave-7f01"
server_key: "580a031281e21e920e7834073c5686fe"
server_key_iv: "780739f8836c4fb629cd9e092c77eb16"
server_key_version: "encryptionkey@0"
I20260808 22:27:06.317546 16588 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-1/wal/instance:
uuid: "616fc1115cca4edc9017bcb25fc8f143"
format_stamp: "Formatted at 2026-08-08 22:27:06 on dist-test-slave-7f01"
server_key: "580a031281e21e920e7834073c5686fe"
server_key_iv: "780739f8836c4fb629cd9e092c77eb16"
server_key_version: "encryptionkey@0"
I20260808 22:27:06.321151 16588 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:27:06.323402 16609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:06.324625 16588 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:27:06.324769 16588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-1/wal
uuid: "616fc1115cca4edc9017bcb25fc8f143"
format_stamp: "Formatted at 2026-08-08 22:27:06 on dist-test-slave-7f01"
server_key: "580a031281e21e920e7834073c5686fe"
server_key_iv: "780739f8836c4fb629cd9e092c77eb16"
server_key_version: "encryptionkey@0"
I20260808 22:27:06.324882 16588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:06.343048 16588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:06.345811 16588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:06.346022 16588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:06.353174 16588 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.253:37661
I20260808 22:27:06.353452 16708 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.253:37661 every 8 connection(s)
I20260808 22:27:06.354144 16588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-1/data/info.pb
I20260808 22:27:06.356887 16709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:27:06.360782 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 16588
I20260808 22:27:06.360924 16709 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 22:27:06 dist-test-slave-7f01 krb5kdc[15754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228026, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.253@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM
Aug 08 22:27:06 dist-test-slave-7f01 krb5kdc[15754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.11.253@KRBTEST.COM
I20260808 22:27:06.374684 15892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.363027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60506 (local address 127.5.11.254:39325)
0808 22:27:06.363196 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:06.363201 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:06.363485 (+ 284us) server_negotiation.cc:407] Connection header received
0808 22:27:06.364007 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:06.364012 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:06.364110 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:06.364215 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:06.365772 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.366280 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.367397 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.367617 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.367700 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:06.370581 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:06.370584 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:06.370587 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:06.370611 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:06.372289 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:06.373010 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:06.373013 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:06.373014 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:06.373090 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:06.373415 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:06.373419 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:06.373421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:06.373731 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:06.373838 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:06.374379 (+ 541us) server_negotiation.cc:299] Negotiation successful
0808 22:27:06.374509 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260808 22:27:06.375587 16713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.362932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.0.0.1:60506)
0808 22:27:06.363335 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:06.363384 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:06.363761 (+ 377us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:06.364368 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:06.364381 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:06.364871 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:06.365625 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:06.365638 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.366409 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.366413 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:06.367237 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:06.367248 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.367797 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.367803 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:06.368045 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:06.368775 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:06.368790 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:06.370337 (+ 1547us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:06.372403 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:06.372411 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:06.372420 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:06.372906 (+ 486us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:06.373173 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:06.373175 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:06.373177 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:06.373307 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:06.373835 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:06.373840 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:06.374148 (+ 308us) client_negotiation.cc:769] Sending connection context
0808 22:27:06.374319 (+ 171us) client_negotiation.cc:240] Negotiation successful
0808 22:27:06.374631 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":117,"thread_start_us":144,"threads_started":1}
I20260808 22:27:06.375584 16712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.362364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38004 (local address 127.5.11.253:37661)
0808 22:27:06.362781 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:06.362794 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:06.362825 (+ 31us) server_negotiation.cc:407] Connection header received
0808 22:27:06.363758 (+ 933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:06.363782 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:06.364011 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:06.365209 (+ 1198us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:06.365410 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.366536 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.367351 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.367825 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.368049 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:06.370966 (+ 2917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:06.370972 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:06.370981 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:06.371002 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:06.372917 (+ 1915us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:06.373520 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:06.373524 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:06.373525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:06.373596 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:06.373896 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:06.373899 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:06.373901 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:06.374189 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:06.374391 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:06.374636 (+ 245us) server_negotiation.cc:299] Negotiation successful
0808 22:27:06.374752 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":89,"server-negotiator.queue_time_us":294,"spinlock_wait_cycles":768,"thread_start_us":154,"threads_started":1}
I20260808 22:27:06.377104 16709 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 22:27:06 dist-test-slave-7f01 krb5kdc[15754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228026, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.253@KRBTEST.COM for oryx/127.5.11.253@KRBTEST.COM
I20260808 22:27:06.384455 16713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.377529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.253:37661 (local address 127.0.0.1:38012)
0808 22:27:06.377617 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:06.377630 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:06.377704 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:06.378135 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:06.378137 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:06.378262 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:06.378346 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:06.378351 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.378998 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.379000 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:06.379488 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:06.379496 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.379941 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.379943 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:06.380046 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:06.380386 (+ 340us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:06.380396 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:06.381557 (+ 1161us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:06.383193 (+ 1636us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:06.383195 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:06.383196 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:06.383391 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:06.383632 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:06.383634 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:06.383635 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:06.383707 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:06.384084 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:06.384086 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:06.384157 (+ 71us) client_negotiation.cc:769] Sending connection context
0808 22:27:06.384234 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 22:27:06.384328 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":15}
I20260808 22:27:06.384524 16712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.377624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38012 (local address 127.5.11.253:37661)
0808 22:27:06.377965 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:06.377968 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:06.377979 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:27:06.378008 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:06.378010 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:06.378042 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:06.378102 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:06.378452 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.378910 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.379586 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.379860 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.379936 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:06.381672 (+ 1736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:06.381675 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:06.381677 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:06.381697 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:06.383093 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:06.383472 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:06.383475 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:06.383476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:06.383554 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:06.383838 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:06.383840 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:06.383841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:06.383989 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:06.384076 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:06.384263 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 22:27:06.384376 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":45}
I20260808 22:27:06.387496 16709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143: Bootstrap starting.
I20260808 22:27:06.389690 16709 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143: Neither blocks nor log segments found. Creating new log.
I20260808 22:27:06.390367 16709 log.cc:824] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143: Log is configured to *not* fsync() on all Append() calls
I20260808 22:27:06.392251 16709 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143: No bootstrap required, opened a new log
I20260808 22:27:06.395093 16709 raft_consensus.cc:348] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } }
I20260808 22:27:06.395277 16709 raft_consensus.cc:374] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:27:06.395309 16709 raft_consensus.cc:729] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 616fc1115cca4edc9017bcb25fc8f143, State: Initialized, Role: FOLLOWER
I20260808 22:27:06.395661 16709 consensus_queue.cc:260] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } }
I20260808 22:27:06.396126 16715 sys_catalog.cc:455] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } } }
I20260808 22:27:06.396261 16715 sys_catalog.cc:458] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:27:06.396778 16709 sys_catalog.cc:564] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 22:27:06.401486 16736 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:27:06.401576 16736 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:27:06.404068 16588 master_runner.cc:428] Detected that this master 616fc1115cca4edc9017bcb25fc8f143 is joining an existing cluster
I20260808 22:27:06.404145 16588 master_runner.cc:432] Initiating AddMaster RPC to add 127.5.11.253:37661
I20260808 22:27:06.416253 16712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.407862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38024 (local address 127.5.11.253:37661)
0808 22:27:06.407991 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:06.407995 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:06.408413 (+ 418us) server_negotiation.cc:407] Connection header received
0808 22:27:06.408544 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:06.408547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:06.408575 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:06.408613 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:06.409148 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.409884 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.410992 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.411267 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.411336 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:06.413005 (+ 1669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:06.413009 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:06.413011 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:06.413030 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:06.414716 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:06.415126 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:06.415129 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:06.415130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:06.415200 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:06.415444 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:06.415446 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:06.415448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:06.415619 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:06.415677 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:06.416004 (+ 327us) server_negotiation.cc:299] Negotiation successful
0808 22:27:06.416111 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 22:27:06.416695 15892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.406334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60512 (local address 127.5.11.254:39325)
0808 22:27:06.407984 (+ 1650us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:06.407989 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:06.408002 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:27:06.408038 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:06.408041 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:06.408079 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:06.408154 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:06.408743 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.409570 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.410799 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.411139 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.411219 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:06.413016 (+ 1797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:06.413020 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:06.413022 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:06.413048 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:06.414874 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:06.415363 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:06.415367 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:06.415369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:06.415453 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:06.415802 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:06.415805 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:06.415807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:06.416009 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:06.416081 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:06.416423 (+ 342us) server_negotiation.cc:299] Negotiation successful
0808 22:27:06.416546 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1577}
I20260808 22:27:06.416718 16745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.406693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.253:37661 (local address 127.0.0.1:38024)
0808 22:27:06.408345 (+ 1652us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:06.408357 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:06.408458 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:06.408718 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:06.408721 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:06.408929 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:06.409041 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:06.409049 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.410001 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.410004 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:06.410873 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:06.410883 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.411469 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.411472 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:06.411572 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:06.412370 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:06.412385 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:06.412883 (+ 498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:06.414823 (+ 1940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:06.414825 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:06.414827 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:06.415040 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:06.415294 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:06.415297 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:06.415299 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:06.415369 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:06.415727 (+ 358us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:06.415729 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:06.415837 (+ 108us) client_negotiation.cc:769] Sending connection context
0808 22:27:06.416435 (+ 598us) client_negotiation.cc:240] Negotiation successful
0808 22:27:06.416564 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1591,"mutex_wait_us":48,"thread_start_us":119,"threads_started":1}
I20260808 22:27:06.417726 16746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.406254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.0.0.1:60512)
0808 22:27:06.406998 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:06.407013 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:06.407101 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:06.408202 (+ 1101us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:06.408205 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:06.408428 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:06.408594 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:06.408602 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.409736 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.409739 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:06.410655 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:06.410667 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.411418 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.411423 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:06.411568 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:06.412288 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:06.412303 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:06.412859 (+ 556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:06.415000 (+ 2141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:06.415004 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:06.415006 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:06.415251 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:06.415580 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:06.415582 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:06.415584 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:06.415668 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:06.416156 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:06.416160 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:06.416245 (+ 85us) client_negotiation.cc:769] Sending connection context
0808 22:27:06.417001 (+ 756us) client_negotiation.cc:240] Negotiation successful
0808 22:27:06.417135 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"mutex_wait_us":1,"spinlock_wait_cycles":24832,"thread_start_us":96,"threads_started":1}
Aug 08 22:27:06 dist-test-slave-7f01 krb5kdc[15754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228025, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.254@KRBTEST.COM for oryx/127.5.11.253@KRBTEST.COM
I20260808 22:27:06.433198 16712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.422873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38038 (local address 127.5.11.253:37661)
0808 22:27:06.422966 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:06.422970 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:06.424058 (+ 1088us) server_negotiation.cc:407] Connection header received
0808 22:27:06.424245 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:06.424249 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:06.424302 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:06.424387 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:06.424955 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.425699 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.426527 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.426869 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.426971 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:06.429487 (+ 2516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:06.429498 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:06.429501 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:06.429531 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:06.431032 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:06.431617 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:06.431623 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:06.431627 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:06.431718 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:06.432101 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:06.432107 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:06.432110 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:06.432331 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:06.432406 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:06.432898 (+ 492us) server_negotiation.cc:299] Negotiation successful
0808 22:27:06.433040 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 22:27:06.433725 16747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.422830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.253:37661 (local address 127.0.0.1:38038)
0808 22:27:06.423923 (+ 1093us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:06.423968 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:06.424118 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:06.424432 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:06.424437 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:06.424736 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:06.424842 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:06.424848 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.425807 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.425809 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:06.426405 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:06.426413 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.426977 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.426980 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:06.427067 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:06.427584 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:06.427596 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:06.429300 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:06.431140 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:06.431150 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:06.431155 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:06.431487 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:06.431873 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:06.431877 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:06.431880 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:06.431978 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:06.432500 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:06.432510 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:06.432644 (+ 134us) client_negotiation.cc:769] Sending connection context
0808 22:27:06.433386 (+ 742us) client_negotiation.cc:240] Negotiation successful
0808 22:27:06.433581 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":870,"thread_start_us":76,"threads_started":1}
I20260808 22:27:06.435127 15823 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.5.11.253:37661: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143"
member_type: NON_VOTER
last_known_addr {
host: "127.5.11.253"
port: 37661
}
attrs {
promote: true
}
}
dest_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6"
cas_config_opid_index: -1
I20260808 22:27:06.435791 15823 consensus_queue.cc:237] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: NON_VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: true } }
I20260808 22:27:06.437729 16751 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 616fc1115cca4edc9017bcb25fc8f143 (127.5.11.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: NON_VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: true } } }
I20260808 22:27:06.438285 16664 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:27:06.439456 16751 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.5.11.253:37661
I20260808 22:27:06.439677 16751 sys_catalog.cc:455] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: NON_VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: true } } }
I20260808 22:27:06.439723 16664 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [term 1 FOLLOWER]: Refusing update from remote peer b62887ed9ca74e17a55fd6afeadb4cf6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260808 22:27:06.439803 16751 sys_catalog.cc:458] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [sys.catalog]: This master's current role is: LEADER
WARNING: no policy specified for oryx/127.5.11.252@KRBTEST.COM; defaulting to no policy
I20260808 22:27:06.440445 16588 master.cc:537] Master@127.5.11.253:37661 shutting down...
I20260808 22:27:06.440928 16750 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: NON_VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.11.252@KRBTEST.COM" created.
I20260808 22:27:06.444957 16588 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:27:06.445255 16588 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:27:06.445367 16588 tablet_replica.cc:330] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143: stopping tablet replica
W20260808 22:27:06.446296 15799 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260808 22:27:06.447227 15799 consensus_peers.cc:487] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 -> Peer 616fc1115cca4edc9017bcb25fc8f143 (127.5.11.253:37661): Couldn't send request to peer 616fc1115cca4edc9017bcb25fc8f143. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 22:27:06.461901 16588 master.cc:552] Master@127.5.11.253:37661 shutdown complete.
I20260808 22:27:06.461968 16588 master_runner.cc:305] Clearing existing system tablet
I20260808 22:27:06.462220 16588 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:06.462299 16588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:06.463498 16588 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:27:06.464459 16762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:06.464658 16588 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:06.464800 16588 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260808 22:27:06.464840 16588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-1/wal
uuid: "616fc1115cca4edc9017bcb25fc8f143"
format_stamp: "Formatted at 2026-08-08 22:27:06 on dist-test-slave-7f01"
server_key: "580a031281e21e920e7834073c5686fe"
server_key_iv: "780739f8836c4fb629cd9e092c77eb16"
server_key_version: "encryptionkey@0"
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.11.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.252.keytab.
Entry for principal oryx/127.5.11.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.252.keytab.
I20260808 22:27:06.466388 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.252:38217
--webserver_interface=127.5.11.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.252.keytab
--principal=oryx/127.5.11.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:46417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:39325,127.5.11.253:37661,127.5.11.252:38217
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:27:06.466807 16588 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:27:06.469144 16588 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 22:27:06.469250 16588 log.cc:1196] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260808 22:27:06.469532 16588 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143: Deleting consensus metadata
I20260808 22:27:06.469679 16588 master_runner.cc:315] Copying system tablet from 127.5.11.254:39325
I20260808 22:27:06.470839 16588 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143: tablet copy: Beginning tablet copy session from remote peer at address 127.5.11.254:39325
I20260808 22:27:06.483808 16775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.471367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.0.0.1:60522)
0808 22:27:06.471690 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:06.471702 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:06.471869 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:06.472151 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:06.472153 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:06.472352 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:06.472454 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:06.472460 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.473220 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.473225 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:06.473819 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:06.473828 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.474302 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.474306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:06.474408 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:06.479887 (+ 5479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:06.479899 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:06.480223 (+ 324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:06.482201 (+ 1978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:06.482205 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:06.482207 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:06.482416 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:06.482737 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:06.482741 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:06.482743 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:06.482830 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:06.483221 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:06.483224 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:06.483329 (+ 105us) client_negotiation.cc:769] Sending connection context
0808 22:27:06.483439 (+ 110us) client_negotiation.cc:240] Negotiation successful
0808 22:27:06.483583 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":111,"threads_started":1}
I20260808 22:27:06.483799 15892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.471433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60522 (local address 127.5.11.254:39325)
0808 22:27:06.471565 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:06.471570 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:06.471812 (+ 242us) server_negotiation.cc:407] Connection header received
0808 22:27:06.471992 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:06.471995 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:06.472050 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:06.472134 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:06.472578 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.473108 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.473953 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.474186 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.474297 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:06.480384 (+ 6087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:06.480387 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:06.480389 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:06.480412 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:06.482068 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:06.482552 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:06.482555 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:06.482557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:06.482642 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:06.482926 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:06.482930 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:06.482932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:06.483120 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:06.483230 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:06.483508 (+ 278us) server_negotiation.cc:299] Negotiation successful
0808 22:27:06.483622 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 22:27:06.484413 15859 tablet_copy_service.cc:140] P b62887ed9ca74e17a55fd6afeadb4cf6: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 616fc1115cca4edc9017bcb25fc8f143 ({username='oryx', principal='oryx/127.5.11.253@KRBTEST.COM'} at 127.0.0.1:60522)
I20260808 22:27:06.484599 15859 tablet_copy_service.cc:161] P b62887ed9ca74e17a55fd6afeadb4cf6: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 616fc1115cca4edc9017bcb25fc8f143 at {username='oryx', principal='oryx/127.5.11.253@KRBTEST.COM'} at 127.0.0.1:60522: session id = 616fc1115cca4edc9017bcb25fc8f143-00000000000000000000000000000000
I20260808 22:27:06.485975 15859 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 22:27:06.487437 16588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 22:27:06.489298 16588 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143: tablet copy: Starting download of 0 data blocks...
I20260808 22:27:06.489518 16588 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143: tablet copy: Starting download of 1 WAL segments...
I20260808 22:27:06.491104 16588 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 22:27:06.492908 15859 tablet_copy_service.cc:342] P b62887ed9ca74e17a55fd6afeadb4cf6: Request end of tablet copy session 616fc1115cca4edc9017bcb25fc8f143-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.5.11.253@KRBTEST.COM'} at 127.0.0.1:60522
I20260808 22:27:06.493031 15859 tablet_copy_service.cc:433] P b62887ed9ca74e17a55fd6afeadb4cf6: ending tablet copy session 616fc1115cca4edc9017bcb25fc8f143-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 616fc1115cca4edc9017bcb25fc8f143
I20260808 22:27:06.500229 16588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:06.501730 16777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:06.504683 16778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:06.504927 16781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:06.505578 16588 server_base.cc:1034] running on GCE node
I20260808 22:27:06.505740 16588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:06.506067 16588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:06.507208 16588 hybrid_clock.cc:648] HybridClock initialized: now 1786228026507200 us; error 24 us; skew 500 ppm
Aug 08 22:27:06 dist-test-slave-7f01 krb5kdc[15754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228026, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:27:06.508672 16588 init.cc:377] Logged in from keytab as oryx/127.5.11.253@KRBTEST.COM (short username oryx)
I20260808 22:27:06.509143 16588 webserver.cc:460] Webserver started at http://127.5.11.253:43177/ using document root <none> and password file <none>
I20260808 22:27:06.509352 16588 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:06.509406 16588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:06.510473 16588 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:27:06.511426 16796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:06.511804 16588 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:27:06.511900 16588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-1/wal
uuid: "616fc1115cca4edc9017bcb25fc8f143"
format_stamp: "Formatted at 2026-08-08 22:27:06 on dist-test-slave-7f01"
server_key: "580a031281e21e920e7834073c5686fe"
server_key_iv: "780739f8836c4fb629cd9e092c77eb16"
server_key_version: "encryptionkey@0"
I20260808 22:27:06.511986 16588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:06.521958 16588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:06.524001 16588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:06.528934 16588 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.253:37661
I20260808 22:27:06.529582 16588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-1/data/info.pb
I20260808 22:27:06.529860 16868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.253:37661 every 8 connection(s)
I20260808 22:27:06.531370 16869 sys_catalog.cc:263] Verifying existing consensus state
I20260808 22:27:06.532342 16869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143: Bootstrap starting.
I20260808 22:27:06.540380 16869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 22:27:06.540707 16869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143: Bootstrap complete.
I20260808 22:27:06.541224 16869 raft_consensus.cc:348] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: NON_VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: true } }
I20260808 22:27:06.541360 16869 raft_consensus.cc:729] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 616fc1115cca4edc9017bcb25fc8f143, State: Initialized, Role: LEARNER
I20260808 22:27:06.541482 16869 consensus_queue.cc:260] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: NON_VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: true } }
I20260808 22:27:06.541826 16874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: NON_VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: true } } }
I20260808 22:27:06.542004 16869 sys_catalog.cc:564] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:27:06.542083 16874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [sys.catalog]: This master's current role is: LEARNER
I20260808 22:27:06.545743 16892 catalog_manager.cc:1259] Loaded cluster ID: e6cced0e5a9e49409b85ee0fb2c9be1c
I20260808 22:27:06.545821 16892 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143: loading cluster ID for follower catalog manager: success
I20260808 22:27:06.547325 16892 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143: acquiring CA information for follower catalog manager: success
I20260808 22:27:06.548512 16892 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260808 22:27:06.622918 16763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:06.623189 16763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:06.623268 16763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:06.627279 16763 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:27:06.627337 16763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:06.627357 16763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:06.627372 16763 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:27:06.627388 16763 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:27:06.632577 16763 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:46417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:39325,127.5.11.253:37661,127.5.11.252:38217
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.252.keytab
--principal=oryx/127.5.11.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.252:38217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.11.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:06.633621 16763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:06.634626 16763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:06.642813 16905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:06.642822 16898 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:06.642831 16900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:06.643213 16763 server_base.cc:1034] running on GCE node
I20260808 22:27:06.643886 16763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:06.644932 16763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:06.646131 16763 hybrid_clock.cc:648] HybridClock initialized: now 1786228026646103 us; error 47 us; skew 500 ppm
Aug 08 22:27:06 dist-test-slave-7f01 krb5kdc[15754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228026, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:27:06.649011 16763 init.cc:377] Logged in from keytab as oryx/127.5.11.252@KRBTEST.COM (short username oryx)
I20260808 22:27:06.649803 16763 webserver.cc:460] Webserver started at http://127.5.11.252:35339/ using document root <none> and password file <none>
I20260808 22:27:06.650313 16763 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:06.650372 16763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:06.650532 16763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:27:06.652626 16763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-2/data/instance:
uuid: "f2de4baf12b64a80b7ee6544655fdb81"
format_stamp: "Formatted at 2026-08-08 22:27:06 on dist-test-slave-7f01"
server_key: "face06ae86491d4414b541d0c16e588c"
server_key_iv: "0694604e3ea4436be0ecb0a4960eb60d"
server_key_version: "encryptionkey@0"
I20260808 22:27:06.653141 16763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-2/wal/instance:
uuid: "f2de4baf12b64a80b7ee6544655fdb81"
format_stamp: "Formatted at 2026-08-08 22:27:06 on dist-test-slave-7f01"
server_key: "face06ae86491d4414b541d0c16e588c"
server_key_iv: "0694604e3ea4436be0ecb0a4960eb60d"
server_key_version: "encryptionkey@0"
I20260808 22:27:06.656664 16763 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:27:06.658888 16914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:06.660132 16763 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:27:06.660255 16763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-2/wal
uuid: "f2de4baf12b64a80b7ee6544655fdb81"
format_stamp: "Formatted at 2026-08-08 22:27:06 on dist-test-slave-7f01"
server_key: "face06ae86491d4414b541d0c16e588c"
server_key_iv: "0694604e3ea4436be0ecb0a4960eb60d"
server_key_version: "encryptionkey@0"
I20260808 22:27:06.660403 16763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:06.671200 16763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:06.674257 16763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:06.674446 16763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:06.681936 16763 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.252:38217
I20260808 22:27:06.681955 16996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.252:38217 every 8 connection(s)
I20260808 22:27:06.682866 16763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-2/data/info.pb
I20260808 22:27:06.684192 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 16763
I20260808 22:27:06.686506 16997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:27:06.690187 16997 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 22:27:06 dist-test-slave-7f01 krb5kdc[15754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.11.252@KRBTEST.COM
Aug 08 22:27:06 dist-test-slave-7f01 krb5kdc[15754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228026, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.252@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM
I20260808 22:27:06.698830 16999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.685527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53382 (local address 127.5.11.252:38217)
0808 22:27:06.685891 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:06.685905 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:06.685939 (+ 34us) server_negotiation.cc:407] Connection header received
0808 22:27:06.686548 (+ 609us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:06.686562 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:06.686937 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:06.687284 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:06.688193 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.689194 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.690074 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.690562 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.690739 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:06.693541 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:06.693548 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:06.693560 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:06.693595 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:06.695574 (+ 1979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:06.696100 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:06.696105 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:06.696109 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:06.696221 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:06.696567 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:06.696572 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:06.696576 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:06.697117 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:06.697312 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:06.697553 (+ 241us) server_negotiation.cc:299] Negotiation successful
0808 22:27:06.697853 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":97,"threads_started":1}
I20260808 22:27:06.702553 17001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.691918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.0.0.1:60524)
0808 22:27:06.692319 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:06.692367 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:06.692565 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:06.693051 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:06.693057 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:06.693388 (+ 331us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:06.693615 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:06.693624 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.694632 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.694636 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:06.695714 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:06.695723 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.696320 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.696323 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:06.696436 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:06.696898 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:06.696911 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:06.698588 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:06.701197 (+ 2609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:06.701204 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:06.701208 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:06.701420 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:06.701658 (+ 238us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:06.701660 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:06.701662 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:06.701735 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:06.702056 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:06.702061 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:06.702168 (+ 107us) client_negotiation.cc:769] Sending connection context
0808 22:27:06.702274 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 22:27:06.702397 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":166,"threads_started":1}
I20260808 22:27:06.702605 15892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.692075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60524 (local address 127.5.11.254:39325)
0808 22:27:06.692213 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:06.692218 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:06.692442 (+ 224us) server_negotiation.cc:407] Connection header received
0808 22:27:06.692697 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:06.692701 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:06.692759 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:06.692822 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:06.693749 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.694480 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.695858 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.696200 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.696291 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:06.698722 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:06.698725 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:06.698727 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:06.698750 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:06.701106 (+ 2356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:06.701506 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:06.701508 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:06.701509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:06.701581 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:06.701809 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:06.701811 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:06.701812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:06.701964 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:06.702046 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:06.702332 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 22:27:06.702461 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 22:27:06.702633 16139 heartbeater.cc:503] Master 127.5.11.254:39325 was elected leader, sending a full tablet report...
I20260808 22:27:06.703810 16997 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 22:27:06 dist-test-slave-7f01 krb5kdc[15754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228026, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.252@KRBTEST.COM for oryx/127.5.11.253@KRBTEST.COM
I20260808 22:27:06.711153 17001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.704269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.253:37661 (local address 127.0.0.1:38052)
0808 22:27:06.704376 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:06.704389 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:06.704490 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:06.704836 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:06.704838 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:06.704962 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:06.705050 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:06.705055 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.705884 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.705886 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:06.706374 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:06.706381 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.706788 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.706791 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:06.706860 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:06.707221 (+ 361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:06.707232 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:06.708404 (+ 1172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:06.709991 (+ 1587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:06.709994 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:06.709995 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:06.710178 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:06.710422 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:06.710424 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:06.710426 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:06.710497 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:06.710823 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:06.710825 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:06.710892 (+ 67us) client_negotiation.cc:769] Sending connection context
0808 22:27:06.710966 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 22:27:06.711053 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 22:27:06.711284 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.704382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38052 (local address 127.5.11.253:37661)
0808 22:27:06.704629 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:06.704641 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:06.704653 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:27:06.704690 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:06.704692 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:06.704729 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:06.704803 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:06.705151 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.705795 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.706468 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.706685 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.706759 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:06.708536 (+ 1777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:06.708540 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:06.708543 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:06.708569 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:06.709892 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:06.710262 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:06.710264 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:06.710265 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:06.710342 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:06.710577 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:06.710580 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:06.710581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:06.710744 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:06.710819 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:06.711028 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 22:27:06.711117 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":103,"threads_started":1}
I20260808 22:27:06.711980 16997 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.11.252" port: 38217 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 22:27:06 dist-test-slave-7f01 krb5kdc[15754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228026, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.252@KRBTEST.COM for oryx/127.5.11.252@KRBTEST.COM
I20260808 22:27:06.719280 17001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.712454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:38217 (local address 127.0.0.1:53398)
0808 22:27:06.712574 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:06.712588 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:06.712659 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:06.712856 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:06.712858 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:06.713023 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:06.713105 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:06.713110 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.714199 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.714202 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:06.714661 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:06.714668 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.715098 (+ 430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.715100 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:06.715156 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:06.715587 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:06.715599 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:06.716724 (+ 1125us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:06.718119 (+ 1395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:06.718122 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:06.718123 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:06.718305 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:06.718548 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:06.718550 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:06.718551 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:06.718623 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:06.718922 (+ 299us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:06.718924 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:06.718992 (+ 68us) client_negotiation.cc:769] Sending connection context
0808 22:27:06.719073 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 22:27:06.719160 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 22:27:06.719383 16999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.712593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53398 (local address 127.5.11.252:38217)
0808 22:27:06.712691 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:06.712694 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:06.712705 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:27:06.712733 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:06.712735 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:06.712765 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:06.712828 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:06.713200 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.714097 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.714755 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.714985 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.715076 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:06.716839 (+ 1763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:06.716842 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:06.716843 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:06.716862 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:06.718020 (+ 1158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:06.718387 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:06.718389 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:06.718390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:06.718464 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:06.718698 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:06.718700 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:06.718702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:06.718844 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:06.718915 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:06.719131 (+ 216us) server_negotiation.cc:299] Negotiation successful
0808 22:27:06.719212 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 22:27:06.722002 16997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81: Bootstrap starting.
I20260808 22:27:06.724287 16997 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81: Neither blocks nor log segments found. Creating new log.
I20260808 22:27:06.725008 16997 log.cc:824] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81: Log is configured to *not* fsync() on all Append() calls
I20260808 22:27:06.726734 16997 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81: No bootstrap required, opened a new log
I20260808 22:27:06.729660 16997 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 38217 } }
I20260808 22:27:06.729842 16997 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:27:06.729920 16997 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2de4baf12b64a80b7ee6544655fdb81, State: Initialized, Role: FOLLOWER
I20260808 22:27:06.730423 16997 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 38217 } }
I20260808 22:27:06.730907 17004 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 38217 } } }
I20260808 22:27:06.731050 17004 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:27:06.731673 16997 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 22:27:06.736585 17018 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:27:06.736670 17018 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:27:06.739868 16763 master_runner.cc:428] Detected that this master f2de4baf12b64a80b7ee6544655fdb81 is joining an existing cluster
I20260808 22:27:06.739919 16763 master_runner.cc:432] Initiating AddMaster RPC to add 127.5.11.252:38217
I20260808 22:27:06.751303 15892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.742301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60532 (local address 127.5.11.254:39325)
0808 22:27:06.742417 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:06.742423 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:06.742441 (+ 18us) server_negotiation.cc:407] Connection header received
0808 22:27:06.742483 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:06.742486 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:06.742538 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:06.742629 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:06.743667 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.744468 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.745774 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.746124 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.746197 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:06.747651 (+ 1454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:06.747655 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:06.747657 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:06.747682 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:06.749449 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:06.749957 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:06.749961 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:06.749963 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:06.750065 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:06.750385 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:06.750388 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:06.750390 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:06.750624 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:06.750736 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:06.751029 (+ 293us) server_negotiation.cc:299] Negotiation successful
0808 22:27:06.751141 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 22:27:06.751870 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 15929
I20260808 22:27:06.751870 16999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.742946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53404 (local address 127.5.11.252:38217)
0808 22:27:06.743053 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:06.743057 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:06.743500 (+ 443us) server_negotiation.cc:407] Connection header received
0808 22:27:06.743666 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:06.743669 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:06.743703 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:06.743768 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:06.744365 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.745048 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.746621 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.746934 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.747006 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:06.748440 (+ 1434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:06.748444 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:06.748446 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:06.748474 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:06.750133 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:06.750570 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:06.750573 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:06.750575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:06.750654 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:06.750996 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:06.750998 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:06.751000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:06.751200 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:06.751265 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:06.751593 (+ 328us) server_negotiation.cc:299] Negotiation successful
0808 22:27:06.751704 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51,"spinlock_wait_cycles":5376}
I20260808 22:27:06.776101 17027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.741851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.0.0.1:60532)
0808 22:27:06.742297 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:06.742312 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:06.742401 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:06.742907 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:06.742912 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:06.743382 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:06.743530 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:06.743539 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.744603 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.744606 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:06.745616 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:06.745631 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.746350 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.746353 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:06.746493 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:06.747019 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:06.747044 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:06.747516 (+ 472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:06.749574 (+ 2058us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:06.749579 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:06.749582 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:06.749844 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:06.750167 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:06.750174 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:06.750178 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:06.750273 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:06.750728 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:06.750731 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:06.750822 (+ 91us) client_negotiation.cc:769] Sending connection context
0808 22:27:06.751674 (+ 852us) client_negotiation.cc:240] Negotiation successful
0808 22:27:06.751833 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":69,"spinlock_wait_cycles":384,"thread_start_us":95,"threads_started":1}
I20260808 22:27:06.777045 17030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.742990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:38217 (local address 127.0.0.1:53404)
0808 22:27:06.743405 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:06.743419 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:06.743572 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:06.743912 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:06.743914 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:06.744151 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:06.744258 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:06.744266 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.745162 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.745164 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:06.746492 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:06.746501 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.747148 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.747152 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:06.747254 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:06.748004 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:06.748018 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:06.748306 (+ 288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:06.750221 (+ 1915us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:06.750224 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:06.750226 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:06.750466 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:06.750755 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:06.750758 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:06.750760 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:06.750899 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:06.751326 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:06.751329 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:06.751407 (+ 78us) client_negotiation.cc:769] Sending connection context
0808 22:27:06.751999 (+ 592us) client_negotiation.cc:240] Negotiation successful
0808 22:27:06.752148 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"mutex_wait_us":53,"thread_start_us":109,"threads_started":1}
I20260808 22:27:06.780616 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.742040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38066 (local address 127.5.11.253:37661)
0808 22:27:06.742270 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:06.742274 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:06.742475 (+ 201us) server_negotiation.cc:407] Connection header received
0808 22:27:06.742644 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:06.742647 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:06.742692 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:06.742777 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:06.743585 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.744422 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.745769 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.746140 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.746220 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:06.776918 (+ 30698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:06.776922 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:06.776926 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:06.776957 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:06.778726 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:06.779252 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:06.779256 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:06.779258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:06.779342 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:06.779640 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:06.779643 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:06.779645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:06.779889 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:06.779964 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:06.780323 (+ 359us) server_negotiation.cc:299] Negotiation successful
0808 22:27:06.780462 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146}
I20260808 22:27:06.781358 17026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.741949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.253:37661 (local address 127.0.0.1:38066)
0808 22:27:06.742398 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:06.742411 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:06.742535 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:06.742811 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:06.742814 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:06.743295 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:06.743437 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:06.743446 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.744582 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.744585 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:06.745627 (+ 1042us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:06.745639 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.746383 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.746386 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:06.746557 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:06.774013 (+ 27456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:06.774031 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:06.776694 (+ 2663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:06.778860 (+ 2166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:06.778864 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:06.778866 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:06.779136 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:06.779451 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:06.779454 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:06.779456 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:06.779540 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:06.780050 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:06.780053 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:06.780145 (+ 92us) client_negotiation.cc:769] Sending connection context
0808 22:27:06.781034 (+ 889us) client_negotiation.cc:240] Negotiation successful
0808 22:27:06.781184 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":388,"mutex_wait_us":130,"thread_start_us":122,"threads_started":1}
I20260808 22:27:06.783525 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.193:39203
--local_ip_for_outbound_sockets=127.5.11.193
--tserver_master_addrs=127.5.11.254:39325
--webserver_port=39243
--webserver_interface=127.5.11.193
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.193.keytab
--principal=oryx/127.5.11.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:46417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.11.254:39325,127.5.11.253:37661
--tserver_master_addrs=127.5.11.254:39325,127.5.11.253:37661,127.5.11.252:38217 with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 22:27:06 dist-test-slave-7f01 krb5kdc[15754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228025, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.254@KRBTEST.COM for oryx/127.5.11.252@KRBTEST.COM
I20260808 22:27:06.791702 16999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.782820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53414 (local address 127.5.11.252:38217)
0808 22:27:06.782946 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:06.782952 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:06.782966 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:27:06.783051 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:06.783055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:06.783103 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:06.783215 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:06.783657 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.784281 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.785128 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.785501 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.785629 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:06.788075 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:06.788079 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:06.788082 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:06.788107 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:06.789775 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:06.790343 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:06.790347 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:06.790349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:06.790437 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:06.790780 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:06.790784 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:06.790786 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:06.790988 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:06.791094 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:06.791447 (+ 353us) server_negotiation.cc:299] Negotiation successful
0808 22:27:06.791558 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 22:27:06.791841 16747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.782272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:38217 (local address 127.0.0.1:53414)
0808 22:27:06.782573 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:06.782590 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:06.782673 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:06.783243 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:06.783246 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:06.783442 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:06.783535 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:06.783541 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.784401 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.784405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:06.784948 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:06.784956 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.785647 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.785651 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:06.785755 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:06.786210 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:06.786225 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:06.787907 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:06.789902 (+ 1995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:06.789908 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:06.789914 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:06.790225 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:06.790546 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:06.790552 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:06.790556 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:06.790671 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:06.791108 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:06.791115 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:06.791247 (+ 132us) client_negotiation.cc:769] Sending connection context
0808 22:27:06.791384 (+ 137us) client_negotiation.cc:240] Negotiation successful
0808 22:27:06.791566 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198}
I20260808 22:27:06.793236 15823 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.5.11.252:38217: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81"
member_type: NON_VOTER
last_known_addr {
host: "127.5.11.252"
port: 38217
}
attrs {
promote: true
}
}
dest_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6"
cas_config_opid_index: 5
I20260808 22:27:06.794036 15823 consensus_queue.cc:237] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: NON_VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: true } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: NON_VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: true } }
I20260808 22:27:06.795377 16750 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f2de4baf12b64a80b7ee6544655fdb81 (127.5.11.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: NON_VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: true } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: NON_VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: true } } }
I20260808 22:27:06.797240 16750 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.5.11.252:38217
I20260808 22:27:06.797413 16750 sys_catalog.cc:455] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: NON_VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: true } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: NON_VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: true } } }
I20260808 22:27:06.797502 16750 sys_catalog.cc:458] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [sys.catalog]: This master's current role is: LEADER
I20260808 22:27:06.797938 16763 master.cc:537] Master@127.5.11.252:38217 shutting down...
W20260808 22:27:06.800815 15803 consensus_peers.cc:487] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 -> Peer f2de4baf12b64a80b7ee6544655fdb81 (127.5.11.252:38217): Couldn't send request to peer f2de4baf12b64a80b7ee6544655fdb81. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260808 22:27:06.801110 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.795725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38080 (local address 127.5.11.253:37661)
0808 22:27:06.796042 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:06.796046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:06.796199 (+ 153us) server_negotiation.cc:407] Connection header received
0808 22:27:06.796367 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:06.796370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:06.796428 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:06.796496 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:27:06.796863 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.797641 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.799439 (+ 1798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.800240 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.800323 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:06.800474 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:06.800832 (+ 358us) server_negotiation.cc:299] Negotiation successful
0808 22:27:06.800980 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":230}
I20260808 22:27:06.801635 16747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.795507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.253:37661 (local address 127.0.0.1:38080)
0808 22:27:06.796133 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:06.796147 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:06.796258 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:06.796643 (+ 385us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:06.796647 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:06.796668 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:27:06.796745 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:06.796750 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.797865 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.797868 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:06.799267 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:06.799282 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.800576 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.800580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:06.800672 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:06.800717 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 22:27:06.801427 (+ 710us) client_negotiation.cc:240] Negotiation successful
0808 22:27:06.801499 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557}
I20260808 22:27:06.802260 16831 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [term 1 LEARNER]: Refusing update from remote peer b62887ed9ca74e17a55fd6afeadb4cf6: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260808 22:27:06.802690 16751 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: NON_VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260808 22:27:06.805735 16831 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER f2de4baf12b64a80b7ee6544655fdb81 (127.5.11.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: NON_VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: true } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: NON_VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: true } } }
I20260808 22:27:06.807907 17034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: NON_VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: true } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: NON_VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: true } } }
I20260808 22:27:06.808037 17034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [sys.catalog]: This master's current role is: LEARNER
I20260808 22:27:06.808071 16874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b62887ed9ca74e17a55fd6afeadb4cf6. Latest consensus state: current_term: 1 leader_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: NON_VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: true } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: NON_VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: true } } }
I20260808 22:27:06.808188 16874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [sys.catalog]: This master's current role is: LEARNER
I20260808 22:27:06.808465 16763 raft_consensus.cc:2229] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260808 22:27:06.808764 16763 raft_consensus.cc:2258] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260808 22:27:06.808864 16763 tablet_replica.cc:330] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81: stopping tablet replica
I20260808 22:27:06.809887 16751 raft_consensus.cc:1050] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6: attempting to promote NON_VOTER 616fc1115cca4edc9017bcb25fc8f143 to VOTER
I20260808 22:27:06.810277 16751 consensus_queue.cc:237] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: false } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: NON_VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: true } }
W20260808 22:27:06.811404 15803 connection.cc:535] client connection to 127.5.11.252:38217 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 22:27:06.812494 17040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.812092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:38217 (local address 127.0.0.1:53426)
0808 22:27:06.812337 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:06.812410 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.252:38217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":83,"threads_started":1}
W20260808 22:27:06.812635 15803 consensus_peers.cc:487] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 -> Peer f2de4baf12b64a80b7ee6544655fdb81 (127.5.11.252:38217): Couldn't send request to peer f2de4baf12b64a80b7ee6544655fdb81. Status: Network error: Client connection negotiation failed: client connection to 127.5.11.252:38217: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 22:27:06.813010 16831 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [term 1 LEARNER]: Refusing update from remote peer b62887ed9ca74e17a55fd6afeadb4cf6: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260808 22:27:06.813453 16750 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260808 22:27:06.815145 16750 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 616fc1115cca4edc9017bcb25fc8f143 (127.5.11.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: false } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: NON_VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: true } } }
I20260808 22:27:06.816383 16831 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 616fc1115cca4edc9017bcb25fc8f143 (127.5.11.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: false } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: NON_VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: true } } }
I20260808 22:27:06.816637 16750 sys_catalog.cc:455] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: false } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: NON_VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: true } } }
I20260808 22:27:06.816746 16750 sys_catalog.cc:458] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [sys.catalog]: This master's current role is: LEADER
I20260808 22:27:06.816875 16750 sys_catalog.cc:455] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: false } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: NON_VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: true } } }
I20260808 22:27:06.816952 16750 sys_catalog.cc:458] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [sys.catalog]: This master's current role is: LEADER
I20260808 22:27:06.818142 16874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: false } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: NON_VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: true } } }
I20260808 22:27:06.818250 16874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:27:06.826392 16763 master.cc:552] Master@127.5.11.252:38217 shutdown complete.
I20260808 22:27:06.826449 16763 master_runner.cc:305] Clearing existing system tablet
I20260808 22:27:06.826691 16763 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:06.826753 16763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:06.828138 16763 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260808 22:27:06.829054 17043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:06.829284 16763 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:06.829413 16763 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:27:06.829447 16763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-2/wal
uuid: "f2de4baf12b64a80b7ee6544655fdb81"
format_stamp: "Formatted at 2026-08-08 22:27:06 on dist-test-slave-7f01"
server_key: "face06ae86491d4414b541d0c16e588c"
server_key_iv: "0694604e3ea4436be0ecb0a4960eb60d"
server_key_version: "encryptionkey@0"
I20260808 22:27:06.831650 16763 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:27:06.834242 16763 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260808 22:27:06.834398 16763 log.cc:1196] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260808 22:27:06.834682 16763 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81: Deleting consensus metadata
I20260808 22:27:06.834851 16763 master_runner.cc:315] Copying system tablet from 127.5.11.254:39325
I20260808 22:27:06.836134 16763 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81: tablet copy: Beginning tablet copy session from remote peer at address 127.5.11.254:39325
I20260808 22:27:06.845970 15892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.837043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60546 (local address 127.5.11.254:39325)
0808 22:27:06.837202 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:06.837210 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:06.837231 (+ 21us) server_negotiation.cc:407] Connection header received
0808 22:27:06.837394 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:06.837398 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:06.837449 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:06.837541 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:06.838117 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.838940 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.840231 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.840569 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.840664 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:06.842212 (+ 1548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:06.842217 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:06.842220 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:06.842247 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:06.844146 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:06.844643 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:06.844647 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:06.844649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:06.844737 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:06.845064 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:06.845068 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:06.845070 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:06.845274 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:06.845343 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:06.845693 (+ 350us) server_negotiation.cc:299] Negotiation successful
0808 22:27:06.845807 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 22:27:06.846647 17052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.836673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.0.0.1:60546)
0808 22:27:06.837140 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:06.837158 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:06.837286 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:06.837580 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:06.837583 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:06.837834 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:06.837976 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:06.837984 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.839089 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.839093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:06.840091 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:06.840103 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:06.840882 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:06.840886 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:06.840977 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:06.841601 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:06.841617 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:06.842057 (+ 440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:06.844277 (+ 2220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:06.844281 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:06.844283 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:06.844530 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:06.844857 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:06.844860 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:06.844862 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:06.844947 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:06.845415 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:06.845418 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:06.845508 (+ 90us) client_negotiation.cc:769] Sending connection context
0808 22:27:06.846309 (+ 801us) client_negotiation.cc:240] Negotiation successful
0808 22:27:06.846461 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":92,"threads_started":1}
I20260808 22:27:06.847107 15859 tablet_copy_service.cc:140] P b62887ed9ca74e17a55fd6afeadb4cf6: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer f2de4baf12b64a80b7ee6544655fdb81 ({username='oryx', principal='oryx/127.5.11.252@KRBTEST.COM'} at 127.0.0.1:60546)
I20260808 22:27:06.847203 15859 tablet_copy_service.cc:161] P b62887ed9ca74e17a55fd6afeadb4cf6: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer f2de4baf12b64a80b7ee6544655fdb81 at {username='oryx', principal='oryx/127.5.11.252@KRBTEST.COM'} at 127.0.0.1:60546: session id = f2de4baf12b64a80b7ee6544655fdb81-00000000000000000000000000000000
I20260808 22:27:06.848294 15859 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6: Tablet Copy: opened 0 blocks and 1 log segments
I20260808 22:27:06.849573 16763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260808 22:27:06.851899 16763 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81: tablet copy: Starting download of 0 data blocks...
I20260808 22:27:06.852115 16763 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81: tablet copy: Starting download of 1 WAL segments...
I20260808 22:27:06.853621 16763 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260808 22:27:06.855320 15859 tablet_copy_service.cc:342] P b62887ed9ca74e17a55fd6afeadb4cf6: Request end of tablet copy session f2de4baf12b64a80b7ee6544655fdb81-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.5.11.252@KRBTEST.COM'} at 127.0.0.1:60546
I20260808 22:27:06.855398 15859 tablet_copy_service.cc:433] P b62887ed9ca74e17a55fd6afeadb4cf6: ending tablet copy session f2de4baf12b64a80b7ee6544655fdb81-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer f2de4baf12b64a80b7ee6544655fdb81
I20260808 22:27:06.863240 16763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:06.864846 17055 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:06.864905 17056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:06.865195 17059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:06.865617 16763 server_base.cc:1034] running on GCE node
I20260808 22:27:06.865762 16763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:06.866026 16763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:06.867221 16763 hybrid_clock.cc:648] HybridClock initialized: now 1786228026867197 us; error 38 us; skew 500 ppm
Aug 08 22:27:06 dist-test-slave-7f01 krb5kdc[15754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228026, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:27:06.868912 16763 init.cc:377] Logged in from keytab as oryx/127.5.11.252@KRBTEST.COM (short username oryx)
I20260808 22:27:06.869393 16763 webserver.cc:460] Webserver started at http://127.5.11.252:39929/ using document root <none> and password file <none>
I20260808 22:27:06.869581 16763 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:06.869625 16763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:06.870884 16763 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:27:06.871933 17070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:06.872180 16763 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260808 22:27:06.872256 16763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-2/wal
uuid: "f2de4baf12b64a80b7ee6544655fdb81"
format_stamp: "Formatted at 2026-08-08 22:27:06 on dist-test-slave-7f01"
server_key: "face06ae86491d4414b541d0c16e588c"
server_key_iv: "0694604e3ea4436be0ecb0a4960eb60d"
server_key_version: "encryptionkey@0"
I20260808 22:27:06.872328 16763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:06.885440 16763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:06.887791 16763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:06.893424 16763 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.252:38217
I20260808 22:27:06.893486 17149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.252:38217 every 8 connection(s)
I20260808 22:27:06.894304 16763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/master-2/data/info.pb
I20260808 22:27:06.896338 17153 sys_catalog.cc:263] Verifying existing consensus state
I20260808 22:27:06.897449 17153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81: Bootstrap starting.
I20260808 22:27:06.909451 17153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 22:27:06.909904 17153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81: Bootstrap complete.
I20260808 22:27:06.910679 17153 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: false } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: NON_VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: true } }
I20260808 22:27:06.910916 17153 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: f2de4baf12b64a80b7ee6544655fdb81, State: Initialized, Role: LEARNER
I20260808 22:27:06.911098 17153 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: false } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: NON_VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: true } }
I20260808 22:27:06.911775 17153 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:27:06.912941 17161 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: false } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: NON_VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: true } } }
I20260808 22:27:06.913100 17161 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [sys.catalog]: This master's current role is: LEARNER
I20260808 22:27:06.916977 17176 catalog_manager.cc:1259] Loaded cluster ID: e6cced0e5a9e49409b85ee0fb2c9be1c
I20260808 22:27:06.917054 17176 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81: loading cluster ID for follower catalog manager: success
I20260808 22:27:06.919211 17176 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81: acquiring CA information for follower catalog manager: success
I20260808 22:27:06.920644 17176 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 22:27:06.929628 16345 heartbeater.cc:503] Master 127.5.11.254:39325 was elected leader, sending a full tablet report...
W20260808 22:27:06.931257 17032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:06.931595 17032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:06.931682 17032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:06.936353 17032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:06.936435 17032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:06.936547 17032 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.193
I20260808 22:27:06.942134 17032 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:46417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.193.keytab
--principal=oryx/127.5.11.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.193:39203
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.11.193
--webserver_port=39243
--tserver_master_addrs=127.5.11.254:39325,127.5.11.253:37661,127.5.11.252:38217
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.193
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:06.943289 17032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:06.944382 17032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:06.953869 17187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:06.953858 17182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:06.953855 17185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:06.954109 17032 server_base.cc:1034] running on GCE node
I20260808 22:27:06.954481 17032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:06.955075 17032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:06.956235 17032 hybrid_clock.cc:648] HybridClock initialized: now 1786228026956209 us; error 36 us; skew 500 ppm
Aug 08 22:27:06 dist-test-slave-7f01 krb5kdc[15754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228026, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:27:06.958890 17032 init.cc:377] Logged in from keytab as oryx/127.5.11.193@KRBTEST.COM (short username oryx)
I20260808 22:27:06.959697 17032 webserver.cc:460] Webserver started at http://127.5.11.193:39243/ using document root <none> and password file <none>
I20260808 22:27:06.960290 17032 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:06.960366 17032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:06.964058 17032 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.003s
I20260808 22:27:06.966051 17196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:06.967255 17032 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:27:06.967391 17032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-0/wal
uuid: "f43f8e31fb01495fabaefe56425dc079"
format_stamp: "Formatted at 2026-08-08 22:27:05 on dist-test-slave-7f01"
server_key: "d567ffe22fe5ff61d01ccb999fd9a258"
server_key_iv: "2142c2d1bfdc3c55f25cf0d9399b6309"
server_key_version: "encryptionkey@0"
I20260808 22:27:06.967949 17032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:06.980214 17032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:06.983397 17032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:06.983598 17032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:06.984202 17032 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:27:06.985046 17032 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:27:06.985095 17032 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:06.985163 17032 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:27:06.985196 17032 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:06.994406 17032 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.193:39203
I20260808 22:27:06.994496 17386 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.193:39203 every 8 connection(s)
I20260808 22:27:06.995434 17032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
I20260808 22:27:07.001837 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 17032
I20260808 22:27:07.002059 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 16149
Aug 08 22:27:07 dist-test-slave-7f01 krb5kdc[15754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228026, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM
Aug 08 22:27:07 dist-test-slave-7f01 krb5kdc[15754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228026, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.252@KRBTEST.COM
Aug 08 22:27:07 dist-test-slave-7f01 krb5kdc[15754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228026, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.253@KRBTEST.COM
I20260808 22:27:07.014863 15892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.997218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:55415 (local address 127.5.11.254:39325)
0808 22:27:06.997430 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:06.997437 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:06.998297 (+ 860us) server_negotiation.cc:407] Connection header received
0808 22:27:06.999542 (+ 1245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:06.999546 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:06.999606 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:06.999663 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:07.001071 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.001880 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.005585 (+ 3705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.006257 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.006362 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:07.009492 (+ 3130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:07.009499 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:07.009504 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:07.009554 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:07.011524 (+ 1970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:07.012426 (+ 902us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:07.012433 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:07.012438 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:07.012552 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:07.013361 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:07.013367 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:07.013369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:07.013639 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:07.013743 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:07.014525 (+ 782us) server_negotiation.cc:299] Negotiation successful
0808 22:27:07.014659 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 22:27:07.016759 17393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.997610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.5.11.193:55415)
0808 22:27:06.998153 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:06.998204 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:06.999314 (+ 1110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:06.999900 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:06.999909 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:07.000386 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:07.000912 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:07.000919 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.002031 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.002034 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:07.005450 (+ 3416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:07.005462 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.006487 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.006490 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:07.006606 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:07.007162 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:07.007183 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:07.009310 (+ 2127us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:07.011653 (+ 2343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:07.011662 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:07.011731 (+ 69us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:07.012220 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:07.012684 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:07.012689 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:07.012694 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:07.013219 (+ 525us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:07.013809 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:07.013817 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:07.014168 (+ 351us) client_negotiation.cc:769] Sending connection context
0808 22:27:07.015047 (+ 879us) client_negotiation.cc:240] Negotiation successful
0808 22:27:07.015613 (+ 566us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":2894,"spinlock_wait_cycles":28288,"thread_start_us":197,"threads_started":1}
I20260808 22:27:07.017246 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.997897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:57431 (local address 127.5.11.253:37661)
0808 22:27:06.998038 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:06.998043 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:06.998310 (+ 267us) server_negotiation.cc:407] Connection header received
0808 22:27:06.999519 (+ 1209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:06.999522 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:06.999587 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:06.999691 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:07.001044 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.001692 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.004117 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.004627 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.004734 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:07.013084 (+ 8350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:07.013090 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:07.013095 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:07.013128 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:07.014866 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:07.015761 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:07.015767 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:07.015768 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:07.015878 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:07.016220 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:07.016226 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:07.016230 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:07.016493 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:07.016572 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:07.016976 (+ 404us) server_negotiation.cc:299] Negotiation successful
0808 22:27:07.017098 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 22:27:07.017918 17394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.997610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.253:37661 (local address 127.5.11.193:57431)
0808 22:27:06.998170 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:06.998205 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:06.999318 (+ 1113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:06.999902 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:06.999910 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:07.000383 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:07.000898 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:07.000912 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.001898 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.001965 (+ 67us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:07.003945 (+ 1980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:07.003961 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.004985 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.004993 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:07.005285 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:07.011342 (+ 6057us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:07.011361 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:07.012831 (+ 1470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:07.014997 (+ 2166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:07.015004 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:07.015008 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:07.015617 (+ 609us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:07.015996 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:07.015999 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:07.016001 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:07.016100 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:07.016654 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:07.016660 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:07.016768 (+ 108us) client_negotiation.cc:769] Sending connection context
0808 22:27:07.017461 (+ 693us) client_negotiation.cc:240] Negotiation successful
0808 22:27:07.017609 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"mutex_wait_us":87,"spinlock_wait_cycles":65024,"thread_start_us":185,"threads_started":1}
I20260808 22:27:07.017968 17395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.998708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:38217 (local address 127.5.11.193:57169)
0808 22:27:06.999020 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:06.999034 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:06.999314 (+ 280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:06.999900 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:06.999909 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:07.000393 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:07.000910 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:07.000919 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.005614 (+ 4695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.005617 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:07.006475 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:07.006483 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.007278 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.007282 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:07.007384 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:07.009963 (+ 2579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:07.009987 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:07.012255 (+ 2268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:07.016296 (+ 4041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:07.016299 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:07.016301 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:07.016544 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:07.016873 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:07.016876 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:07.016878 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:07.016973 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:07.017445 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:07.017448 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:07.017556 (+ 108us) client_negotiation.cc:769] Sending connection context
0808 22:27:07.017649 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 22:27:07.017799 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":146,"spinlock_wait_cycles":6016,"thread_start_us":133,"threads_started":1}
I20260808 22:27:07.018486 17396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:06.998874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:57169 (local address 127.5.11.252:38217)
0808 22:27:06.999145 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:06.999149 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:06.999163 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:27:06.999616 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:06.999619 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:06.999674 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:06.999778 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:07.001055 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.002125 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.006604 (+ 4479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.007023 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.007139 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:07.012428 (+ 5289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:07.012432 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:07.012435 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:07.012463 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:07.016168 (+ 3705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:07.016663 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:07.016666 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:07.016668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:07.016761 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:07.017083 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:07.017086 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:07.017088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:07.017282 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:07.017399 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:07.018180 (+ 781us) server_negotiation.cc:299] Negotiation successful
0808 22:27:07.018308 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":124,"threads_started":1}
I20260808 22:27:07.018472 17390 heartbeater.cc:347] Connected to a master server at 127.5.11.254:39325
I20260808 22:27:07.018734 17390 heartbeater.cc:464] Registering TS with master...
I20260808 22:27:07.018740 17391 heartbeater.cc:347] Connected to a master server at 127.5.11.253:37661
I20260808 22:27:07.018877 17391 heartbeater.cc:464] Registering TS with master...
I20260808 22:27:07.019187 17389 heartbeater.cc:347] Connected to a master server at 127.5.11.252:38217
I20260808 22:27:07.019279 17389 heartbeater.cc:464] Registering TS with master...
I20260808 22:27:07.019338 17391 heartbeater.cc:511] Master 127.5.11.253:37661 requested a full tablet report, sending...
I20260808 22:27:07.019340 17390 heartbeater.cc:511] Master 127.5.11.254:39325 requested a full tablet report, sending...
I20260808 22:27:07.019425 17389 heartbeater.cc:511] Master 127.5.11.252:38217 requested a full tablet report, sending...
I20260808 22:27:07.020735 15823 ts_manager.cc:194] Re-registered known tserver with Master: f43f8e31fb01495fabaefe56425dc079 (127.5.11.193:39203)
I20260808 22:27:07.020929 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.194:43785
--local_ip_for_outbound_sockets=127.5.11.194
--tserver_master_addrs=127.5.11.254:39325
--webserver_port=46407
--webserver_interface=127.5.11.194
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.194.keytab
--principal=oryx/127.5.11.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:46417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.11.254:39325,127.5.11.253:37661
--tserver_master_addrs=127.5.11.254:39325,127.5.11.253:37661,127.5.11.252:38217 with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:27:07.020980 17095 ts_manager.cc:194] Registered new tserver with Master: f43f8e31fb01495fabaefe56425dc079 (127.5.11.193:39203)
I20260808 22:27:07.020982 16820 ts_manager.cc:194] Registered new tserver with Master: f43f8e31fb01495fabaefe56425dc079 (127.5.11.193:39203)
I20260808 22:27:07.021472 15823 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.11.193@KRBTEST.COM'} at 127.5.11.193:55415
W20260808 22:27:07.135380 17400 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:07.135604 17400 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:07.135645 17400 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:07.139577 17400 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:07.139632 17400 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:07.139688 17400 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.194
I20260808 22:27:07.143461 16577 heartbeater.cc:503] Master 127.5.11.254:39325 was elected leader, sending a full tablet report...
I20260808 22:27:07.145081 17400 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:46417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.194.keytab
--principal=oryx/127.5.11.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.194:43785
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.11.194
--webserver_port=46407
--tserver_master_addrs=127.5.11.254:39325,127.5.11.253:37661,127.5.11.252:38217
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17400
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.194
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:07.145957 17400 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:07.146907 17400 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:07.156548 17412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:07.156566 17416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:07.156589 17413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:07.156911 17400 server_base.cc:1034] running on GCE node
I20260808 22:27:07.157310 17400 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:07.157980 17400 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:07.159180 17400 hybrid_clock.cc:648] HybridClock initialized: now 1786228027159137 us; error 61 us; skew 500 ppm
Aug 08 22:27:07 dist-test-slave-7f01 krb5kdc[15754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228027, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:27:07.162303 17400 init.cc:377] Logged in from keytab as oryx/127.5.11.194@KRBTEST.COM (short username oryx)
I20260808 22:27:07.163197 17400 webserver.cc:460] Webserver started at http://127.5.11.194:46407/ using document root <none> and password file <none>
I20260808 22:27:07.163841 17400 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:07.163951 17400 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:07.167565 17400 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.001s
I20260808 22:27:07.169822 17429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:07.171097 17400 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:27:07.171234 17400 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-1/wal
uuid: "bc97c1ed9ee6470b99e8b230c5c84b61"
format_stamp: "Formatted at 2026-08-08 22:27:05 on dist-test-slave-7f01"
server_key: "5a2b0e738d9803a25b8fd64da8848658"
server_key_iv: "b4a427ff5555803c4ba3d5be2a993d1b"
server_key_version: "encryptionkey@0"
I20260808 22:27:07.171794 17400 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:07.188898 17400 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:07.192137 17400 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:07.192351 17400 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:07.192926 17400 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:27:07.193838 17400 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:27:07.193912 17400 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:07.193982 17400 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:27:07.194026 17400 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:07.203449 17400 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.194:43785
I20260808 22:27:07.203507 17617 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.194:43785 every 8 connection(s)
I20260808 22:27:07.204502 17400 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
I20260808 22:27:07.207940 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 17400
I20260808 22:27:07.208163 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 16366
Aug 08 22:27:07 dist-test-slave-7f01 krb5kdc[15754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228027, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.253@KRBTEST.COM
Aug 08 22:27:07 dist-test-slave-7f01 krb5kdc[15754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228027, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.252@KRBTEST.COM
I20260808 22:27:07.221279 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:07.206924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:46321 (local address 127.5.11.253:37661)
0808 22:27:07.207053 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:07.207057 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:07.207945 (+ 888us) server_negotiation.cc:407] Connection header received
0808 22:27:07.209273 (+ 1328us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:07.209279 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:07.209345 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:07.209399 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:07.211032 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.211732 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.213321 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.213668 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.213744 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:07.216955 (+ 3211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:07.216963 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:07.216969 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:07.217001 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:07.218796 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:07.219454 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:07.219460 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:07.219464 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:07.219547 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:07.219966 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:07.219971 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:07.219975 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:07.220211 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:07.220279 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:07.220922 (+ 643us) server_negotiation.cc:299] Negotiation successful
0808 22:27:07.221054 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 22:27:07.222996 17625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:07.206701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.253:37661 (local address 127.5.11.194:46321)
0808 22:27:07.207823 (+ 1122us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:07.207838 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:07.209042 (+ 1204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:07.209779 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:07.209789 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:07.210265 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:07.210837 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:07.210865 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.211906 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.211912 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:07.213181 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:07.213194 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.213965 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.213971 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:07.214189 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:07.214865 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:07.214889 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:07.216761 (+ 1872us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:07.218928 (+ 2167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:07.218936 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:07.218953 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:07.219349 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:07.219656 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:07.219661 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:07.219665 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:07.219860 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:07.220356 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:07.220364 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:07.220654 (+ 290us) client_negotiation.cc:769] Sending connection context
0808 22:27:07.221632 (+ 978us) client_negotiation.cc:240] Negotiation successful
0808 22:27:07.221985 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1039,"mutex_wait_us":97,"spinlock_wait_cycles":23936,"thread_start_us":132,"threads_started":1}
I20260808 22:27:07.224951 17396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:07.206281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:41463 (local address 127.5.11.252:38217)
0808 22:27:07.206460 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:07.206466 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:07.211311 (+ 4845us) server_negotiation.cc:407] Connection header received
0808 22:27:07.211578 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:07.211585 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:07.211665 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:07.211764 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:07.212759 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.213939 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.215717 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.216266 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.216384 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:07.219968 (+ 3584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:07.219975 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:07.219981 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:07.220022 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:07.222548 (+ 2526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:07.223226 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:07.223233 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:07.223238 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:07.223361 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:07.223831 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:07.223837 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:07.223841 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:07.224089 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:07.224185 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:07.224588 (+ 403us) server_negotiation.cc:299] Negotiation successful
0808 22:27:07.224732 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260808 22:27:07.225046 17623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:07.206701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:38217 (local address 127.5.11.194:41463)
0808 22:27:07.211202 (+ 4501us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:07.211221 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:07.211397 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:07.212010 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:07.212013 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:07.212360 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:07.212575 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:07.212588 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.214223 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.214229 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:07.215534 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:07.215551 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.216615 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.216622 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:07.216771 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:07.217720 (+ 949us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:07.217742 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:07.219768 (+ 2026us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:07.222722 (+ 2954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:07.222728 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:07.222733 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:07.223073 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:07.223525 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:07.223532 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:07.223537 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:07.223667 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:07.224216 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:07.224222 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:07.224369 (+ 147us) client_negotiation.cc:769] Sending connection context
0808 22:27:07.224529 (+ 160us) client_negotiation.cc:240] Negotiation successful
0808 22:27:07.224723 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4396,"mutex_wait_us":381,"thread_start_us":136,"threads_started":1}
I20260808 22:27:07.229836 17619 heartbeater.cc:347] Connected to a master server at 127.5.11.252:38217
I20260808 22:27:07.230119 17619 heartbeater.cc:464] Registering TS with master...
I20260808 22:27:07.230687 17621 heartbeater.cc:347] Connected to a master server at 127.5.11.253:37661
I20260808 22:27:07.230803 17621 heartbeater.cc:464] Registering TS with master...
I20260808 22:27:07.230798 17619 heartbeater.cc:511] Master 127.5.11.252:38217 requested a full tablet report, sending...
I20260808 22:27:07.230993 17621 heartbeater.cc:511] Master 127.5.11.253:37661 requested a full tablet report, sending...
I20260808 22:27:07.232219 16820 ts_manager.cc:194] Registered new tserver with Master: bc97c1ed9ee6470b99e8b230c5c84b61 (127.5.11.194:43785)
I20260808 22:27:07.232219 17095 ts_manager.cc:194] Registered new tserver with Master: bc97c1ed9ee6470b99e8b230c5c84b61 (127.5.11.194:43785)
Aug 08 22:27:07 dist-test-slave-7f01 krb5kdc[15754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228027, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM
I20260808 22:27:07.237732 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.195:45457
--local_ip_for_outbound_sockets=127.5.11.195
--tserver_master_addrs=127.5.11.254:39325
--webserver_port=38479
--webserver_interface=127.5.11.195
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.195.keytab
--principal=oryx/127.5.11.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:46417
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.5.11.254:39325,127.5.11.253:37661
--tserver_master_addrs=127.5.11.254:39325,127.5.11.253:37661,127.5.11.252:38217 with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:27:07.239846 17624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:07.206700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.5.11.194:59681)
0808 22:27:07.207407 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:07.207459 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:07.229740 (+ 22281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:07.230224 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:07.230230 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:07.230526 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:07.230925 (+ 399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:07.230937 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.232146 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.232152 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:07.233216 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:07.233231 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.233694 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.233700 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:07.233828 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:07.234406 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:07.234419 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:07.236001 (+ 1582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:07.238035 (+ 2034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:07.238041 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:07.238045 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:07.238311 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:07.238621 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:07.238626 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:07.238631 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:07.238739 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:07.239173 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:07.239180 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:07.239302 (+ 122us) client_negotiation.cc:769] Sending connection context
0808 22:27:07.239449 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 22:27:07.239627 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"mutex_wait_us":1,"thread_start_us":157,"threads_started":1}
I20260808 22:27:07.240300 15892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:07.207017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:59681 (local address 127.5.11.254:39325)
0808 22:27:07.207147 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:07.207152 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:07.207557 (+ 405us) server_negotiation.cc:407] Connection header received
0808 22:27:07.229909 (+ 22352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:07.229916 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:07.229987 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:07.230047 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:07.231085 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.231988 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.233332 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.233555 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.233640 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:07.236148 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:07.236152 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:07.236154 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:07.236191 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:07.237902 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:07.238429 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:07.238432 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:07.238433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:07.238525 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:07.238850 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:07.238853 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:07.238854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:07.239039 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:07.239118 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:07.239531 (+ 413us) server_negotiation.cc:299] Negotiation successful
0808 22:27:07.239680 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 22:27:07.241083 17620 heartbeater.cc:347] Connected to a master server at 127.5.11.254:39325
I20260808 22:27:07.241205 17620 heartbeater.cc:464] Registering TS with master...
I20260808 22:27:07.241408 17620 heartbeater.cc:511] Master 127.5.11.254:39325 requested a full tablet report, sending...
I20260808 22:27:07.242156 15823 ts_manager.cc:194] Re-registered known tserver with Master: bc97c1ed9ee6470b99e8b230c5c84b61 (127.5.11.194:43785)
I20260808 22:27:07.243067 15823 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.11.194@KRBTEST.COM'} at 127.5.11.194:59681
W20260808 22:27:07.352703 17627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:07.352941 17627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:07.352979 17627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:07.356896 17627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:07.356949 17627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:07.357050 17627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.195
I20260808 22:27:07.361946 17627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:46417
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.195.keytab
--principal=oryx/127.5.11.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.195:45457
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.11.195
--webserver_port=38479
--tserver_master_addrs=127.5.11.254:39325,127.5.11.253:37661,127.5.11.252:38217
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.195
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:07.362882 17627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:07.363890 17627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:07.373489 17638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:07.373524 17637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:07.373555 17640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:07.373804 17627 server_base.cc:1034] running on GCE node
I20260808 22:27:07.374212 17627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:07.374897 17627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:07.376132 17627 hybrid_clock.cc:648] HybridClock initialized: now 1786228027376111 us; error 27 us; skew 500 ppm
Aug 08 22:27:07 dist-test-slave-7f01 krb5kdc[15754](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228027, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:27:07.378809 17627 init.cc:377] Logged in from keytab as oryx/127.5.11.195@KRBTEST.COM (short username oryx)
I20260808 22:27:07.379626 17627 webserver.cc:460] Webserver started at http://127.5.11.195:38479/ using document root <none> and password file <none>
I20260808 22:27:07.380204 17627 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:07.380319 17627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:07.384227 17627 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:27:07.386173 17654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:07.387333 17627 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:27:07.387470 17627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-2/wal
uuid: "feed9713203a46bfb1c3a49ba5cecd66"
format_stamp: "Formatted at 2026-08-08 22:27:06 on dist-test-slave-7f01"
server_key: "6216b51792a0f620b8c46b5910d4a7b6"
server_key_iv: "31ababbe7d430b00b69ea534ce7cad73"
server_key_version: "encryptionkey@0"
I20260808 22:27:07.388057 17627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:07.423195 17663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:07.419394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:38217 (local address 127.0.0.1:47454)
0808 22:27:07.419631 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:07.419650 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:07.419806 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:07.420109 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:07.420111 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:07.420127 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:27:07.420220 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:07.420225 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.421189 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.421191 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:07.421960 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:07.421969 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.422862 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.422865 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:07.422939 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:07.422961 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 22:27:07.423014 (+ 53us) client_negotiation.cc:240] Negotiation successful
0808 22:27:07.423077 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":86,"threads_started":1}
I20260808 22:27:07.423214 17396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:07.419469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47454 (local address 127.5.11.252:38217)
0808 22:27:07.419600 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:07.419605 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:07.419721 (+ 116us) server_negotiation.cc:407] Connection header received
0808 22:27:07.419912 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:07.419915 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:07.419980 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:07.420088 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:27:07.420320 (+ 232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.421091 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.422084 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.422746 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.422844 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:07.422987 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:07.423029 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 22:27:07.423088 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 22:27:07.424144 17114 raft_consensus.cc:1203] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260808 22:27:07.424723 17664 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b62887ed9ca74e17a55fd6afeadb4cf6. Latest consensus state: current_term: 1 leader_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: false } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: NON_VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: true } } }
I20260808 22:27:07.424911 17664 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [sys.catalog]: This master's current role is: LEARNER
I20260808 22:27:07.438277 17627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:07.441200 17627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:07.441395 17627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:07.442080 17627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:27:07.443048 17627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:27:07.443138 17627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:07.443220 17627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:27:07.443251 17627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:07.452096 17627 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.195:45457
I20260808 22:27:07.452140 17831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.195:45457 every 8 connection(s)
I20260808 22:27:07.453104 17627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
I20260808 22:27:07.456436 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 17627
Aug 08 22:27:07 dist-test-slave-7f01 krb5kdc[15754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228027, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.253@KRBTEST.COM
Aug 08 22:27:07 dist-test-slave-7f01 krb5kdc[15754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228027, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM
Aug 08 22:27:07 dist-test-slave-7f01 krb5kdc[15754](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228027, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.252@KRBTEST.COM
I20260808 22:27:07.472438 15892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:07.456648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:46591 (local address 127.5.11.254:39325)
0808 22:27:07.456768 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:07.456772 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:07.457025 (+ 253us) server_negotiation.cc:407] Connection header received
0808 22:27:07.457215 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:07.457218 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:07.457267 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:07.457316 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:07.459548 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.460317 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.462795 (+ 2478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.463464 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.463568 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:07.466967 (+ 3399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:07.466972 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:07.466977 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:07.467016 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:07.469224 (+ 2208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:07.469793 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:07.469799 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:07.469803 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:07.469892 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:07.470325 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:07.470328 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:07.470330 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:07.470590 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:07.470698 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:07.472146 (+ 1448us) server_negotiation.cc:299] Negotiation successful
0808 22:27:07.472270 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":51}
I20260808 22:27:07.472445 17396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:07.454880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:40475 (local address 127.5.11.252:38217)
0808 22:27:07.455064 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:07.455070 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:07.455985 (+ 915us) server_negotiation.cc:407] Connection header received
0808 22:27:07.457161 (+ 1176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:07.457166 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:07.457228 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:07.457282 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:07.460231 (+ 2949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.460775 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.464227 (+ 3452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.464620 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.464713 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:07.467645 (+ 2932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:07.467649 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:07.467652 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:07.467676 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:07.469807 (+ 2131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:07.470885 (+ 1078us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:07.470891 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:07.470893 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:07.470985 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:07.471363 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:07.471367 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:07.471369 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:07.471595 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:07.471666 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:07.472107 (+ 441us) server_negotiation.cc:299] Negotiation successful
0808 22:27:07.472238 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 22:27:07.473547 17838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:07.456344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.5.11.195:46591)
0808 22:27:07.456937 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:07.456950 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:07.457109 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:07.457691 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:07.457700 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:07.458305 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:07.459120 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:07.459145 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.460950 (+ 1805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.460953 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:07.462674 (+ 1721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:07.462685 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.464107 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.464112 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:07.464224 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:07.464705 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:07.464720 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:07.466797 (+ 2077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:07.469334 (+ 2537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:07.469337 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:07.469339 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:07.469673 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:07.470007 (+ 334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:07.470011 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:07.470014 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:07.470215 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:07.470767 (+ 552us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:07.470775 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:07.471109 (+ 334us) client_negotiation.cc:769] Sending connection context
0808 22:27:07.471328 (+ 219us) client_negotiation.cc:240] Negotiation successful
0808 22:27:07.471697 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"mutex_wait_us":911,"thread_start_us":105,"threads_started":1}
I20260808 22:27:07.474179 17836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:07.455302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:38217 (local address 127.5.11.195:40475)
0808 22:27:07.455824 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:07.455877 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:07.456922 (+ 1045us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:07.457691 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:07.457700 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:07.458381 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:07.459671 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:07.459679 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.460888 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.460894 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:07.461771 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:07.461782 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.464871 (+ 3089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.464877 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:07.464977 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:07.465449 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:07.465471 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:07.467492 (+ 2021us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:07.470188 (+ 2696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:07.470193 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:07.470195 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:07.470757 (+ 562us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:07.471136 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:07.471139 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:07.471143 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:07.471243 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:07.471761 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:07.471764 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:07.471859 (+ 95us) client_negotiation.cc:769] Sending connection context
0808 22:27:07.473125 (+ 1266us) client_negotiation.cc:240] Negotiation successful
0808 22:27:07.473295 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"mutex_wait_us":70,"thread_start_us":147,"threads_started":1}
I20260808 22:27:07.474632 17848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:07.458698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54266 (local address 127.5.11.253:37661)
0808 22:27:07.459912 (+ 1214us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:07.459917 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:07.459929 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:27:07.459962 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:07.459965 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:07.460004 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:07.460091 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:07.461065 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.461783 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.462919 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.463280 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.463396 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:07.465999 (+ 2603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:07.466003 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:07.466008 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:07.466036 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:07.469077 (+ 3041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:07.470031 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:07.470034 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:07.470036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:07.470116 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:07.472233 (+ 2117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:07.472240 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:07.472242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:07.473758 (+ 1516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:07.473903 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:07.474229 (+ 326us) server_negotiation.cc:299] Negotiation successful
0808 22:27:07.474348 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1145,"thread_start_us":70,"threads_started":1}
I20260808 22:27:07.475504 17832 heartbeater.cc:347] Connected to a master server at 127.5.11.252:38217
I20260808 22:27:07.475538 17833 heartbeater.cc:347] Connected to a master server at 127.5.11.254:39325
I20260808 22:27:07.475787 17833 heartbeater.cc:464] Registering TS with master...
I20260808 22:27:07.475788 17832 heartbeater.cc:464] Registering TS with master...
I20260808 22:27:07.476430 17832 heartbeater.cc:511] Master 127.5.11.252:38217 requested a full tablet report, sending...
I20260808 22:27:07.476518 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:07.455543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:48659 (local address 127.5.11.253:37661)
0808 22:27:07.455671 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:07.455675 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:07.456742 (+ 1067us) server_negotiation.cc:407] Connection header received
0808 22:27:07.457166 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:07.457170 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:07.457227 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:07.457281 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:07.459298 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.460051 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.461818 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.462122 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.462196 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:07.465475 (+ 3279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:07.465498 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:07.465502 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:07.465533 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:07.468456 (+ 2923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:07.474966 (+ 6510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:07.474969 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:07.474971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:07.475082 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:07.475425 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:07.475428 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:07.475430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:07.475647 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:07.475732 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:07.476242 (+ 510us) server_negotiation.cc:299] Negotiation successful
0808 22:27:07.476359 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 22:27:07.476972 17855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:07.459365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47470 (local address 127.5.11.252:38217)
0808 22:27:07.461168 (+ 1803us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:07.461172 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:07.463880 (+ 2708us) server_negotiation.cc:407] Connection header received
0808 22:27:07.463923 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:07.463927 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:07.463985 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:07.464073 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:07.465195 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.466782 (+ 1587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.468970 (+ 2188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.469365 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.469465 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:07.471315 (+ 1850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:07.471319 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:07.471322 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:07.471347 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:07.473225 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:07.474564 (+ 1339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:07.474568 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:07.474570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:07.474662 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:07.475871 (+ 1209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:07.475875 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:07.475877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:07.476096 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:07.476223 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:07.476714 (+ 491us) server_negotiation.cc:299] Negotiation successful
0808 22:27:07.476844 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1733,"thread_start_us":58,"threads_started":1}
I20260808 22:27:07.476431 17833 heartbeater.cc:511] Master 127.5.11.254:39325 requested a full tablet report, sending...
I20260808 22:27:07.477269 17839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:07.455302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.253:37661 (local address 127.5.11.195:48659)
0808 22:27:07.456645 (+ 1343us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:07.456664 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:07.456935 (+ 271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:07.457691 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:07.457700 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:07.458293 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:07.459122 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:07.459145 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.460255 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.460259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:07.461683 (+ 1424us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:07.461694 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.462389 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.462392 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:07.462591 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:07.463208 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:07.463229 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:07.465265 (+ 2036us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:07.469217 (+ 3952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:07.469223 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:07.469238 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:07.474649 (+ 5411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:07.475209 (+ 560us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:07.475213 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:07.475215 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:07.475314 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:07.475933 (+ 619us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:07.475936 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:07.476043 (+ 107us) client_negotiation.cc:769] Sending connection context
0808 22:27:07.476788 (+ 745us) client_negotiation.cc:240] Negotiation successful
0808 22:27:07.476926 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1254,"mutex_wait_us":125,"spinlock_wait_cycles":14720,"thread_start_us":129,"threads_started":1}
I20260808 22:27:07.477722 17094 ts_manager.cc:194] Registered new tserver with Master: feed9713203a46bfb1c3a49ba5cecd66 (127.5.11.195:45457)
I20260808 22:27:07.477841 17834 heartbeater.cc:347] Connected to a master server at 127.5.11.253:37661
I20260808 22:27:07.477931 17834 heartbeater.cc:464] Registering TS with master...
I20260808 22:27:07.478103 17834 heartbeater.cc:511] Master 127.5.11.253:37661 requested a full tablet report, sending...
I20260808 22:27:07.478425 15823 ts_manager.cc:194] Re-registered known tserver with Master: feed9713203a46bfb1c3a49ba5cecd66 (127.5.11.195:45457)
I20260808 22:27:07.478711 16820 ts_manager.cc:194] Registered new tserver with Master: feed9713203a46bfb1c3a49ba5cecd66 (127.5.11.195:45457)
I20260808 22:27:07.479035 15823 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.11.195@KRBTEST.COM'} at 127.5.11.195:46591
I20260808 22:27:07.480866 17849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:07.458707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51354 (local address 127.5.11.254:39325)
0808 22:27:07.459077 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:07.459081 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:07.461390 (+ 2309us) server_negotiation.cc:407] Connection header received
0808 22:27:07.461562 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:07.461565 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:07.461607 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:07.461667 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:07.462657 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.463669 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.467969 (+ 4300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.468322 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.468421 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:07.471766 (+ 3345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:07.471770 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:07.471772 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:07.471796 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:07.473981 (+ 2185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:07.479576 (+ 5595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:07.479580 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:07.479581 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:07.479675 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:07.479995 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:07.479999 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:07.480001 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:07.480184 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:07.480326 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:07.480566 (+ 240us) server_negotiation.cc:299] Negotiation successful
0808 22:27:07.480695 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":219,"server-negotiator.queue_time_us":305,"thread_start_us":75,"threads_started":1}
I20260808 22:27:07.483443 15823 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51354:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:27:07.486466 15823 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:27:07.489147 17628 sys_catalog.cc:455] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: false } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: NON_VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: true } } }
I20260808 22:27:07.489291 17628 sys_catalog.cc:458] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [sys.catalog]: This master's current role is: LEADER
I20260808 22:27:07.492730 17628 raft_consensus.cc:1050] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6: attempting to promote NON_VOTER f2de4baf12b64a80b7ee6544655fdb81 to VOTER
I20260808 22:27:07.493080 17628 consensus_queue.cc:237] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: false } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: false } }
I20260808 22:27:07.494859 17114 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [term 1 LEARNER]: Refusing update from remote peer b62887ed9ca74e17a55fd6afeadb4cf6: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 22:27:07.495326 17858 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260808 22:27:07.495831 16831 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [term 1 FOLLOWER]: Refusing update from remote peer b62887ed9ca74e17a55fd6afeadb4cf6: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260808 22:27:07.496793 17868 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260808 22:27:07.498253 17858 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, f2de4baf12b64a80b7ee6544655fdb81 (127.5.11.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: false } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: false } } }
I20260808 22:27:07.499105 16831 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, f2de4baf12b64a80b7ee6544655fdb81 (127.5.11.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: false } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: false } } }
I20260808 22:27:07.499811 17858 sys_catalog.cc:455] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: false } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: false } } }
I20260808 22:27:07.499938 17858 sys_catalog.cc:458] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [sys.catalog]: This master's current role is: LEADER
I20260808 22:27:07.499888 17857 sys_catalog.cc:455] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: false } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: false } } }
I20260808 22:27:07.499993 17857 sys_catalog.cc:458] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [sys.catalog]: This master's current role is: LEADER
I20260808 22:27:07.500402 17114 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, f2de4baf12b64a80b7ee6544655fdb81 (127.5.11.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: false } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: false } } }
I20260808 22:27:07.500674 17857 sys_catalog.cc:455] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: false } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: false } } }
I20260808 22:27:07.500783 17857 sys_catalog.cc:458] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6 [sys.catalog]: This master's current role is: LEADER
I20260808 22:27:07.501381 17872 sys_catalog.cc:455] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: false } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: false } } }
I20260808 22:27:07.501631 17872 sys_catalog.cc:458] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:27:07.503633 17664 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: false } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: false } } }
I20260808 22:27:07.503778 17664 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:27:07.508306 17663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:07.502194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:39203 (local address 127.0.0.1:42674)
0808 22:27:07.502531 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:07.502549 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:07.502647 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:07.504152 (+ 1505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:07.504157 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:07.504175 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:27:07.504279 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:07.504286 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.505917 (+ 1631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.505922 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:07.506987 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:07.507007 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.507899 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.507902 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:07.507986 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:07.508020 (+ 34us) client_negotiation.cc:769] Sending connection context
0808 22:27:07.508086 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 22:27:07.508173 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":62}
I20260808 22:27:07.508664 17878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:07.502815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42674 (local address 127.5.11.193:39203)
0808 22:27:07.503638 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:07.503654 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:07.503676 (+ 22us) server_negotiation.cc:407] Connection header received
0808 22:27:07.503771 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:07.503776 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:07.503984 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:07.504174 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:27:07.504429 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.505781 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.507141 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.507713 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.507905 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:07.508075 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:07.508128 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 22:27:07.508216 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":726,"thread_start_us":103,"threads_started":1}
I20260808 22:27:07.508827 17876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:07.502500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:45457 (local address 127.0.0.1:55072)
0808 22:27:07.503303 (+ 803us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:07.503317 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:07.503406 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:07.504326 (+ 920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:07.504329 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:07.504348 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:27:07.504455 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:07.504461 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.505891 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.505895 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:07.507112 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:07.507123 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.508400 (+ 1277us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.508403 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:07.508532 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:07.508558 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 22:27:07.508633 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 22:27:07.508699 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":735,"thread_start_us":103,"threads_started":1}
I20260808 22:27:07.508924 17879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:07.502913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55072 (local address 127.5.11.195:45457)
0808 22:27:07.503407 (+ 494us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:07.503419 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:07.503444 (+ 25us) server_negotiation.cc:407] Connection header received
0808 22:27:07.503535 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:07.503541 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:07.504067 (+ 526us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:07.504189 (+ 122us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:27:07.504622 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.505763 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.507251 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.508111 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.508202 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:07.508332 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:07.508683 (+ 351us) server_negotiation.cc:299] Negotiation successful
0808 22:27:07.508776 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":394,"thread_start_us":121,"threads_started":1}
I20260808 22:27:07.509785 17873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:07.502804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:43785 (local address 127.0.0.1:42404)
0808 22:27:07.503854 (+ 1050us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:07.503868 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:07.503948 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:07.505137 (+ 1189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:07.505140 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:07.505157 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:27:07.505239 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:07.505244 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.506601 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.506605 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:07.507711 (+ 1106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:07.507722 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.509389 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.509393 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:07.509479 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:07.509503 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 22:27:07.509584 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 22:27:07.509655 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":993,"thread_start_us":101,"threads_started":1}
I20260808 22:27:07.509979 17880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:07.502936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42404 (local address 127.5.11.194:43785)
0808 22:27:07.504478 (+ 1542us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:07.504492 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:07.504514 (+ 22us) server_negotiation.cc:407] Connection header received
0808 22:27:07.504616 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:07.504621 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:07.504832 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:07.504972 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:27:07.505364 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.506468 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.507845 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.509254 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.509366 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:07.509512 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:07.509608 (+ 96us) server_negotiation.cc:299] Negotiation successful
0808 22:27:07.509696 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1436,"thread_start_us":109,"threads_started":1}
I20260808 22:27:07.511866 17276 tablet_service.cc:1467] Processing CreateTablet for tablet 32f9d3d9714c452a94106578eace2b69 (DEFAULT_TABLE table=test-table [id=feea3c53e9eb446aa20398e90c836673]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:27:07.511883 17720 tablet_service.cc:1467] Processing CreateTablet for tablet 32f9d3d9714c452a94106578eace2b69 (DEFAULT_TABLE table=test-table [id=feea3c53e9eb446aa20398e90c836673]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:27:07.512975 17276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32f9d3d9714c452a94106578eace2b69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:27:07.512815 17514 tablet_service.cc:1467] Processing CreateTablet for tablet 32f9d3d9714c452a94106578eace2b69 (DEFAULT_TABLE table=test-table [id=feea3c53e9eb446aa20398e90c836673]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:27:07.513249 17720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32f9d3d9714c452a94106578eace2b69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:27:07.513794 17514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32f9d3d9714c452a94106578eace2b69. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:27:07.520022 17881 tablet_bootstrap.cc:492] T 32f9d3d9714c452a94106578eace2b69 P f43f8e31fb01495fabaefe56425dc079: Bootstrap starting.
I20260808 22:27:07.521809 17882 tablet_bootstrap.cc:492] T 32f9d3d9714c452a94106578eace2b69 P feed9713203a46bfb1c3a49ba5cecd66: Bootstrap starting.
I20260808 22:27:07.522733 17881 tablet_bootstrap.cc:654] T 32f9d3d9714c452a94106578eace2b69 P f43f8e31fb01495fabaefe56425dc079: Neither blocks nor log segments found. Creating new log.
I20260808 22:27:07.523452 17881 log.cc:824] T 32f9d3d9714c452a94106578eace2b69 P f43f8e31fb01495fabaefe56425dc079: Log is configured to *not* fsync() on all Append() calls
I20260808 22:27:07.523620 17882 tablet_bootstrap.cc:654] T 32f9d3d9714c452a94106578eace2b69 P feed9713203a46bfb1c3a49ba5cecd66: Neither blocks nor log segments found. Creating new log.
I20260808 22:27:07.524063 17883 tablet_bootstrap.cc:492] T 32f9d3d9714c452a94106578eace2b69 P bc97c1ed9ee6470b99e8b230c5c84b61: Bootstrap starting.
I20260808 22:27:07.524487 17882 log.cc:824] T 32f9d3d9714c452a94106578eace2b69 P feed9713203a46bfb1c3a49ba5cecd66: Log is configured to *not* fsync() on all Append() calls
I20260808 22:27:07.525743 17881 tablet_bootstrap.cc:492] T 32f9d3d9714c452a94106578eace2b69 P f43f8e31fb01495fabaefe56425dc079: No bootstrap required, opened a new log
I20260808 22:27:07.525941 17881 ts_tablet_manager.cc:1397] T 32f9d3d9714c452a94106578eace2b69 P f43f8e31fb01495fabaefe56425dc079: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s
I20260808 22:27:07.526154 17883 tablet_bootstrap.cc:654] T 32f9d3d9714c452a94106578eace2b69 P bc97c1ed9ee6470b99e8b230c5c84b61: Neither blocks nor log segments found. Creating new log.
I20260808 22:27:07.526337 17882 tablet_bootstrap.cc:492] T 32f9d3d9714c452a94106578eace2b69 P feed9713203a46bfb1c3a49ba5cecd66: No bootstrap required, opened a new log
I20260808 22:27:07.526552 17882 ts_tablet_manager.cc:1397] T 32f9d3d9714c452a94106578eace2b69 P feed9713203a46bfb1c3a49ba5cecd66: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 22:27:07.527194 17883 log.cc:824] T 32f9d3d9714c452a94106578eace2b69 P bc97c1ed9ee6470b99e8b230c5c84b61: Log is configured to *not* fsync() on all Append() calls
I20260808 22:27:07.529217 17883 tablet_bootstrap.cc:492] T 32f9d3d9714c452a94106578eace2b69 P bc97c1ed9ee6470b99e8b230c5c84b61: No bootstrap required, opened a new log
I20260808 22:27:07.529387 17883 ts_tablet_manager.cc:1397] T 32f9d3d9714c452a94106578eace2b69 P bc97c1ed9ee6470b99e8b230c5c84b61: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 22:27:07.530270 17881 raft_consensus.cc:348] T 32f9d3d9714c452a94106578eace2b69 P f43f8e31fb01495fabaefe56425dc079 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f43f8e31fb01495fabaefe56425dc079" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39203 } } peers { permanent_uuid: "feed9713203a46bfb1c3a49ba5cecd66" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 45457 } } peers { permanent_uuid: "bc97c1ed9ee6470b99e8b230c5c84b61" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 43785 } }
I20260808 22:27:07.530551 17881 raft_consensus.cc:374] T 32f9d3d9714c452a94106578eace2b69 P f43f8e31fb01495fabaefe56425dc079 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:27:07.530602 17881 raft_consensus.cc:729] T 32f9d3d9714c452a94106578eace2b69 P f43f8e31fb01495fabaefe56425dc079 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f43f8e31fb01495fabaefe56425dc079, State: Initialized, Role: FOLLOWER
I20260808 22:27:07.530856 17882 raft_consensus.cc:348] T 32f9d3d9714c452a94106578eace2b69 P feed9713203a46bfb1c3a49ba5cecd66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f43f8e31fb01495fabaefe56425dc079" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39203 } } peers { permanent_uuid: "feed9713203a46bfb1c3a49ba5cecd66" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 45457 } } peers { permanent_uuid: "bc97c1ed9ee6470b99e8b230c5c84b61" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 43785 } }
I20260808 22:27:07.531087 17882 raft_consensus.cc:374] T 32f9d3d9714c452a94106578eace2b69 P feed9713203a46bfb1c3a49ba5cecd66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:27:07.531184 17882 raft_consensus.cc:729] T 32f9d3d9714c452a94106578eace2b69 P feed9713203a46bfb1c3a49ba5cecd66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: feed9713203a46bfb1c3a49ba5cecd66, State: Initialized, Role: FOLLOWER
I20260808 22:27:07.531157 17881 consensus_queue.cc:260] T 32f9d3d9714c452a94106578eace2b69 P f43f8e31fb01495fabaefe56425dc079 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f43f8e31fb01495fabaefe56425dc079" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39203 } } peers { permanent_uuid: "feed9713203a46bfb1c3a49ba5cecd66" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 45457 } } peers { permanent_uuid: "bc97c1ed9ee6470b99e8b230c5c84b61" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 43785 } }
I20260808 22:27:07.531626 17882 consensus_queue.cc:260] T 32f9d3d9714c452a94106578eace2b69 P feed9713203a46bfb1c3a49ba5cecd66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f43f8e31fb01495fabaefe56425dc079" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39203 } } peers { permanent_uuid: "feed9713203a46bfb1c3a49ba5cecd66" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 45457 } } peers { permanent_uuid: "bc97c1ed9ee6470b99e8b230c5c84b61" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 43785 } }
I20260808 22:27:07.532164 17881 ts_tablet_manager.cc:1428] T 32f9d3d9714c452a94106578eace2b69 P f43f8e31fb01495fabaefe56425dc079: Time spent starting tablet: real 0.006s user 0.000s sys 0.006s
I20260808 22:27:07.532496 17882 ts_tablet_manager.cc:1428] T 32f9d3d9714c452a94106578eace2b69 P feed9713203a46bfb1c3a49ba5cecd66: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260808 22:27:07.532547 17390 heartbeater.cc:503] Master 127.5.11.254:39325 was elected leader, sending a full tablet report...
I20260808 22:27:07.533903 17883 raft_consensus.cc:348] T 32f9d3d9714c452a94106578eace2b69 P bc97c1ed9ee6470b99e8b230c5c84b61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f43f8e31fb01495fabaefe56425dc079" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39203 } } peers { permanent_uuid: "feed9713203a46bfb1c3a49ba5cecd66" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 45457 } } peers { permanent_uuid: "bc97c1ed9ee6470b99e8b230c5c84b61" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 43785 } }
I20260808 22:27:07.534166 17883 raft_consensus.cc:374] T 32f9d3d9714c452a94106578eace2b69 P bc97c1ed9ee6470b99e8b230c5c84b61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:27:07.534224 17883 raft_consensus.cc:729] T 32f9d3d9714c452a94106578eace2b69 P bc97c1ed9ee6470b99e8b230c5c84b61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc97c1ed9ee6470b99e8b230c5c84b61, State: Initialized, Role: FOLLOWER
I20260808 22:27:07.534646 17883 consensus_queue.cc:260] T 32f9d3d9714c452a94106578eace2b69 P bc97c1ed9ee6470b99e8b230c5c84b61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f43f8e31fb01495fabaefe56425dc079" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39203 } } peers { permanent_uuid: "feed9713203a46bfb1c3a49ba5cecd66" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 45457 } } peers { permanent_uuid: "bc97c1ed9ee6470b99e8b230c5c84b61" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 43785 } }
I20260808 22:27:07.535188 17833 heartbeater.cc:503] Master 127.5.11.254:39325 was elected leader, sending a full tablet report...
I20260808 22:27:07.535604 17883 ts_tablet_manager.cc:1428] T 32f9d3d9714c452a94106578eace2b69 P bc97c1ed9ee6470b99e8b230c5c84b61: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 22:27:07.538067 17620 heartbeater.cc:503] Master 127.5.11.254:39325 was elected leader, sending a full tablet report...
I20260808 22:27:07.580302 17890 raft_consensus.cc:482] T 32f9d3d9714c452a94106578eace2b69 P feed9713203a46bfb1c3a49ba5cecd66 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:27:07.580544 17890 raft_consensus.cc:504] T 32f9d3d9714c452a94106578eace2b69 P feed9713203a46bfb1c3a49ba5cecd66 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f43f8e31fb01495fabaefe56425dc079" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39203 } } peers { permanent_uuid: "feed9713203a46bfb1c3a49ba5cecd66" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 45457 } } peers { permanent_uuid: "bc97c1ed9ee6470b99e8b230c5c84b61" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 43785 } }
I20260808 22:27:07.581797 17890 leader_election.cc:288] T 32f9d3d9714c452a94106578eace2b69 P feed9713203a46bfb1c3a49ba5cecd66 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f43f8e31fb01495fabaefe56425dc079 (127.5.11.193:39203), bc97c1ed9ee6470b99e8b230c5c84b61 (127.5.11.194:43785)
I20260808 22:27:07.585354 17878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:07.582154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:50233 (local address 127.5.11.193:39203)
0808 22:27:07.582263 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:07.582269 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:07.582313 (+ 44us) server_negotiation.cc:407] Connection header received
0808 22:27:07.582367 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:07.582370 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:07.582428 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:07.582536 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:27:07.582800 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.583319 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.584413 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.584801 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.584929 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:07.584980 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:07.585168 (+ 188us) server_negotiation.cc:299] Negotiation successful
0808 22:27:07.585236 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 22:27:07.585372 17839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:07.582015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:39203 (local address 127.5.11.195:50233)
0808 22:27:07.582145 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:07.582161 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:07.582251 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:07.582544 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:07.582547 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:07.582562 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:27:07.582690 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:07.582696 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.583434 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.583439 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:07.584284 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:07.584293 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.584921 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.584924 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:07.585041 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:07.585060 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 22:27:07.585150 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 22:27:07.585221 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 22:27:07.585543 17836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:07.582125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:43785 (local address 127.5.11.195:40779)
0808 22:27:07.582257 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:07.582273 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:07.582375 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:07.582782 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:07.582785 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:07.582802 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:27:07.582915 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:07.582920 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.583700 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.583704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:07.584477 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:07.584487 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.585120 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.585124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:07.585234 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:07.585263 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 22:27:07.585356 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 22:27:07.585424 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":62}
I20260808 22:27:07.585563 17880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:07.582437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:40779 (local address 127.5.11.194:43785)
0808 22:27:07.582573 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:07.582577 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:07.582589 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:27:07.582634 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:07.582636 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:07.582675 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:07.582767 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:27:07.583029 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.583577 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.584615 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.584997 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.585126 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:07.585176 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:07.585376 (+ 200us) server_negotiation.cc:299] Negotiation successful
0808 22:27:07.585447 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 22:27:07.586160 17549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32f9d3d9714c452a94106578eace2b69" candidate_uuid: "feed9713203a46bfb1c3a49ba5cecd66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc97c1ed9ee6470b99e8b230c5c84b61" is_pre_election: true
I20260808 22:27:07.586160 17308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32f9d3d9714c452a94106578eace2b69" candidate_uuid: "feed9713203a46bfb1c3a49ba5cecd66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f43f8e31fb01495fabaefe56425dc079" is_pre_election: true
I20260808 22:27:07.586452 17308 raft_consensus.cc:2454] T 32f9d3d9714c452a94106578eace2b69 P f43f8e31fb01495fabaefe56425dc079 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate feed9713203a46bfb1c3a49ba5cecd66 in term 0.
I20260808 22:27:07.586463 17549 raft_consensus.cc:2454] T 32f9d3d9714c452a94106578eace2b69 P bc97c1ed9ee6470b99e8b230c5c84b61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate feed9713203a46bfb1c3a49ba5cecd66 in term 0.
I20260808 22:27:07.586930 17657 leader_election.cc:302] T 32f9d3d9714c452a94106578eace2b69 P feed9713203a46bfb1c3a49ba5cecd66 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f43f8e31fb01495fabaefe56425dc079, feed9713203a46bfb1c3a49ba5cecd66; no voters:
I20260808 22:27:07.587244 17890 raft_consensus.cc:2781] T 32f9d3d9714c452a94106578eace2b69 P feed9713203a46bfb1c3a49ba5cecd66 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:27:07.587342 17890 raft_consensus.cc:482] T 32f9d3d9714c452a94106578eace2b69 P feed9713203a46bfb1c3a49ba5cecd66 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:27:07.587390 17890 raft_consensus.cc:3037] T 32f9d3d9714c452a94106578eace2b69 P feed9713203a46bfb1c3a49ba5cecd66 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:27:07.588482 17890 raft_consensus.cc:504] T 32f9d3d9714c452a94106578eace2b69 P feed9713203a46bfb1c3a49ba5cecd66 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f43f8e31fb01495fabaefe56425dc079" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39203 } } peers { permanent_uuid: "feed9713203a46bfb1c3a49ba5cecd66" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 45457 } } peers { permanent_uuid: "bc97c1ed9ee6470b99e8b230c5c84b61" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 43785 } }
I20260808 22:27:07.588819 17890 leader_election.cc:288] T 32f9d3d9714c452a94106578eace2b69 P feed9713203a46bfb1c3a49ba5cecd66 [CANDIDATE]: Term 1 election: Requested vote from peers f43f8e31fb01495fabaefe56425dc079 (127.5.11.193:39203), bc97c1ed9ee6470b99e8b230c5c84b61 (127.5.11.194:43785)
I20260808 22:27:07.589166 17308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32f9d3d9714c452a94106578eace2b69" candidate_uuid: "feed9713203a46bfb1c3a49ba5cecd66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f43f8e31fb01495fabaefe56425dc079"
I20260808 22:27:07.589192 17549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32f9d3d9714c452a94106578eace2b69" candidate_uuid: "feed9713203a46bfb1c3a49ba5cecd66" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc97c1ed9ee6470b99e8b230c5c84b61"
I20260808 22:27:07.589298 17308 raft_consensus.cc:3037] T 32f9d3d9714c452a94106578eace2b69 P f43f8e31fb01495fabaefe56425dc079 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:27:07.589315 17549 raft_consensus.cc:3037] T 32f9d3d9714c452a94106578eace2b69 P bc97c1ed9ee6470b99e8b230c5c84b61 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:27:07.590354 17549 raft_consensus.cc:2454] T 32f9d3d9714c452a94106578eace2b69 P bc97c1ed9ee6470b99e8b230c5c84b61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate feed9713203a46bfb1c3a49ba5cecd66 in term 1.
I20260808 22:27:07.590366 17308 raft_consensus.cc:2454] T 32f9d3d9714c452a94106578eace2b69 P f43f8e31fb01495fabaefe56425dc079 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate feed9713203a46bfb1c3a49ba5cecd66 in term 1.
I20260808 22:27:07.590658 17658 leader_election.cc:302] T 32f9d3d9714c452a94106578eace2b69 P feed9713203a46bfb1c3a49ba5cecd66 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc97c1ed9ee6470b99e8b230c5c84b61, feed9713203a46bfb1c3a49ba5cecd66; no voters:
I20260808 22:27:07.590867 17890 raft_consensus.cc:2781] T 32f9d3d9714c452a94106578eace2b69 P feed9713203a46bfb1c3a49ba5cecd66 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:27:07.591073 17890 raft_consensus.cc:686] T 32f9d3d9714c452a94106578eace2b69 P feed9713203a46bfb1c3a49ba5cecd66 [term 1 LEADER]: Becoming Leader. State: Replica: feed9713203a46bfb1c3a49ba5cecd66, State: Running, Role: LEADER
I20260808 22:27:07.591423 17890 consensus_queue.cc:237] T 32f9d3d9714c452a94106578eace2b69 P feed9713203a46bfb1c3a49ba5cecd66 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f43f8e31fb01495fabaefe56425dc079" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39203 } } peers { permanent_uuid: "feed9713203a46bfb1c3a49ba5cecd66" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 45457 } } peers { permanent_uuid: "bc97c1ed9ee6470b99e8b230c5c84b61" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 43785 } }
I20260808 22:27:07.595875 15823 catalog_manager.cc:5495] T 32f9d3d9714c452a94106578eace2b69 P feed9713203a46bfb1c3a49ba5cecd66 reported cstate change: term changed from 0 to 1, leader changed from <none> to feed9713203a46bfb1c3a49ba5cecd66 (127.5.11.195). New cstate: current_term: 1 leader_uuid: "feed9713203a46bfb1c3a49ba5cecd66" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f43f8e31fb01495fabaefe56425dc079" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "feed9713203a46bfb1c3a49ba5cecd66" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 45457 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc97c1ed9ee6470b99e8b230c5c84b61" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 43785 } health_report { overall_health: UNKNOWN } } }
I20260808 22:27:07.625478 17879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:07.622234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55082 (local address 127.5.11.195:45457)
0808 22:27:07.622366 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:07.622370 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:07.622383 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:27:07.622515 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:07.622517 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:07.622575 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:07.622665 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:27:07.622890 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.623379 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.624200 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.624406 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.624483 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:07.624751 (+ 268us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:27:07.625157 (+ 406us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:27:07.625249 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:07.625321 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 22:27:07.625369 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 22:27:07.631505 17308 raft_consensus.cc:1261] T 32f9d3d9714c452a94106578eace2b69 P f43f8e31fb01495fabaefe56425dc079 [term 1 FOLLOWER]: Refusing update from remote peer feed9713203a46bfb1c3a49ba5cecd66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:27:07.631505 17549 raft_consensus.cc:1261] T 32f9d3d9714c452a94106578eace2b69 P bc97c1ed9ee6470b99e8b230c5c84b61 [term 1 FOLLOWER]: Refusing update from remote peer feed9713203a46bfb1c3a49ba5cecd66: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:27:07.632326 17892 consensus_queue.cc:1035] T 32f9d3d9714c452a94106578eace2b69 P feed9713203a46bfb1c3a49ba5cecd66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc97c1ed9ee6470b99e8b230c5c84b61" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 43785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:27:07.632521 17890 consensus_queue.cc:1035] T 32f9d3d9714c452a94106578eace2b69 P feed9713203a46bfb1c3a49ba5cecd66 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f43f8e31fb01495fabaefe56425dc079" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:27:07.644165 17901 mvcc.cc:205] Tried to move back new op lower bound from 7316390001170530304 to 7316390001017872384. Current Snapshot: MvccSnapshot[applied={T|T < 7316390001170530304}]
I20260808 22:27:07.647519 17896 mvcc.cc:205] Tried to move back new op lower bound from 7316390001170530304 to 7316390001017872384. Current Snapshot: MvccSnapshot[applied={T|T < 7316390001170530304 or (T in {7316390001170530304})}]
I20260808 22:27:07.652175 15823 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51354:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:27:07.652343 15823 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51354:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:27:07.657768 15823 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b62887ed9ca74e17a55fd6afeadb4cf6: Sending DeleteTablet for 3 replicas of tablet 32f9d3d9714c452a94106578eace2b69
I20260808 22:27:07.658463 17276 tablet_service.cc:1514] Processing DeleteTablet for tablet 32f9d3d9714c452a94106578eace2b69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:27:07 UTC) from {username='oryx', principal='oryx/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:42674
I20260808 22:27:07.658551 17720 tablet_service.cc:1514] Processing DeleteTablet for tablet 32f9d3d9714c452a94106578eace2b69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:27:07 UTC) from {username='oryx', principal='oryx/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:55072
I20260808 22:27:07.658715 17514 tablet_service.cc:1514] Processing DeleteTablet for tablet 32f9d3d9714c452a94106578eace2b69 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:27:07 UTC) from {username='oryx', principal='oryx/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:42404
I20260808 22:27:07.658970 17910 tablet_replica.cc:330] T 32f9d3d9714c452a94106578eace2b69 P f43f8e31fb01495fabaefe56425dc079: stopping tablet replica
I20260808 22:27:07.659190 17913 tablet_replica.cc:330] T 32f9d3d9714c452a94106578eace2b69 P bc97c1ed9ee6470b99e8b230c5c84b61: stopping tablet replica
I20260808 22:27:07.659296 17910 raft_consensus.cc:2229] T 32f9d3d9714c452a94106578eace2b69 P f43f8e31fb01495fabaefe56425dc079 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:27:07.659600 17913 raft_consensus.cc:2229] T 32f9d3d9714c452a94106578eace2b69 P bc97c1ed9ee6470b99e8b230c5c84b61 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:27:07.659616 17910 raft_consensus.cc:2258] T 32f9d3d9714c452a94106578eace2b69 P f43f8e31fb01495fabaefe56425dc079 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:27:07.660415 17913 raft_consensus.cc:2258] T 32f9d3d9714c452a94106578eace2b69 P bc97c1ed9ee6470b99e8b230c5c84b61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:27:07.661069 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 15776
I20260808 22:27:07.670850 17910 ts_tablet_manager.cc:1905] T 32f9d3d9714c452a94106578eace2b69 P f43f8e31fb01495fabaefe56425dc079: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:27:07.670849 17913 ts_tablet_manager.cc:1905] T 32f9d3d9714c452a94106578eace2b69 P bc97c1ed9ee6470b99e8b230c5c84b61: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:27:07.673986 17910 ts_tablet_manager.cc:1918] T 32f9d3d9714c452a94106578eace2b69 P f43f8e31fb01495fabaefe56425dc079: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:27:07.674124 17910 log.cc:1196] T 32f9d3d9714c452a94106578eace2b69 P f43f8e31fb01495fabaefe56425dc079: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-0/wal/wals/32f9d3d9714c452a94106578eace2b69
I20260808 22:27:07.674486 17910 ts_tablet_manager.cc:1939] T 32f9d3d9714c452a94106578eace2b69 P f43f8e31fb01495fabaefe56425dc079: Deleting consensus metadata
I20260808 22:27:07.674474 17913 ts_tablet_manager.cc:1918] T 32f9d3d9714c452a94106578eace2b69 P bc97c1ed9ee6470b99e8b230c5c84b61: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:27:07.674739 17913 log.cc:1196] T 32f9d3d9714c452a94106578eace2b69 P bc97c1ed9ee6470b99e8b230c5c84b61: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-1/wal/wals/32f9d3d9714c452a94106578eace2b69
I20260808 22:27:07.675128 17913 ts_tablet_manager.cc:1939] T 32f9d3d9714c452a94106578eace2b69 P bc97c1ed9ee6470b99e8b230c5c84b61: Deleting consensus metadata
I20260808 22:27:07.676136 17911 tablet_replica.cc:330] T 32f9d3d9714c452a94106578eace2b69 P feed9713203a46bfb1c3a49ba5cecd66: stopping tablet replica
I20260808 22:27:07.676467 17911 raft_consensus.cc:2229] T 32f9d3d9714c452a94106578eace2b69 P feed9713203a46bfb1c3a49ba5cecd66 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:27:07.676851 17911 raft_consensus.cc:2258] T 32f9d3d9714c452a94106578eace2b69 P feed9713203a46bfb1c3a49ba5cecd66 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:27:07.678453 17911 ts_tablet_manager.cc:1905] T 32f9d3d9714c452a94106578eace2b69 P feed9713203a46bfb1c3a49ba5cecd66: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:27:07.681221 17911 ts_tablet_manager.cc:1918] T 32f9d3d9714c452a94106578eace2b69 P feed9713203a46bfb1c3a49ba5cecd66: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:27:07.681330 17911 log.cc:1196] T 32f9d3d9714c452a94106578eace2b69 P feed9713203a46bfb1c3a49ba5cecd66: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-2/wal/wals/32f9d3d9714c452a94106578eace2b69
I20260808 22:27:07.681706 17911 ts_tablet_manager.cc:1939] T 32f9d3d9714c452a94106578eace2b69 P feed9713203a46bfb1c3a49ba5cecd66: Deleting consensus metadata
W20260808 22:27:07.682840 17442 connection.cc:535] server connection from 127.0.0.1:42404 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 22:27:07.683249 17660 connection.cc:535] server connection from 127.0.0.1:55072 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260808 22:27:07.683256 17203 connection.cc:535] server connection from 127.0.0.1:42674 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 22:27:07.688182 17002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:07.660184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54272 (local address 127.5.11.253:37661)
0808 22:27:07.660314 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:07.660320 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:07.675394 (+ 15074us) server_negotiation.cc:407] Connection header received
0808 22:27:07.676331 (+ 937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:07.676335 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:07.676390 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:07.676446 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:07.678376 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.679190 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.680488 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.680834 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.680917 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:07.683482 (+ 2565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:07.683486 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:07.683489 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:07.683529 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:07.685571 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:07.686100 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:07.686104 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:07.686106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:07.686205 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:07.686518 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:07.686521 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:07.686523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:07.686743 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:07.686925 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:07.687858 (+ 933us) server_negotiation.cc:299] Negotiation successful
0808 22:27:07.687977 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 22:27:07.698871 17396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:07.688853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47490 (local address 127.5.11.252:38217)
0808 22:27:07.688976 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:07.688981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:07.689179 (+ 198us) server_negotiation.cc:407] Connection header received
0808 22:27:07.689241 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:07.689244 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:07.689284 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:07.689359 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:07.690264 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.691032 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.692401 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.692738 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.692820 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:07.695110 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:07.695114 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:07.695117 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:07.695142 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:07.696983 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:07.697456 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:07.697459 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:07.697461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:07.697542 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:07.697889 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:07.697892 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:07.697894 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:07.698085 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:07.698189 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:07.698530 (+ 341us) server_negotiation.cc:299] Negotiation successful
0808 22:27:07.698671 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 22:27:07.699363 17848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:07.687234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54280 (local address 127.5.11.253:37661)
0808 22:27:07.687346 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:07.687350 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:07.687555 (+ 205us) server_negotiation.cc:407] Connection header received
0808 22:27:07.687713 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:07.687716 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:07.687795 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:07.687866 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:07.689863 (+ 1997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.690658 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.692120 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.692480 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.692571 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:07.695047 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:07.695054 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:07.695060 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:07.695090 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:07.697503 (+ 2413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:07.697994 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:07.698000 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:07.698005 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:07.698092 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:07.698395 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:07.698398 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:07.698400 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:07.698645 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:07.698741 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:07.699088 (+ 347us) server_negotiation.cc:299] Negotiation successful
0808 22:27:07.699208 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 22:27:07.699698 17855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:07.660722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47482 (local address 127.5.11.252:38217)
0808 22:27:07.660833 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:07.660839 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:07.688213 (+ 27374us) server_negotiation.cc:407] Connection header received
0808 22:27:07.688264 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:07.688267 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:07.688321 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:07.688392 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:07.689865 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.690658 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.692120 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:07.692485 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:07.692576 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:07.695141 (+ 2565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:07.695145 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:07.695147 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:07.695170 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:07.697341 (+ 2171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:07.697851 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:07.697855 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:07.697857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:07.697948 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:07.698258 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:07.698261 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:07.698262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:07.698471 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:07.698621 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:07.699225 (+ 604us) server_negotiation.cc:299] Negotiation successful
0808 22:27:07.699332 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 22:27:08.646723 17926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:08.646113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.5.11.193:41753)
0808 22:27:08.646476 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:08.646604 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":90,"threads_started":1}
I20260808 22:27:08.646718 17925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:08.646113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.5.11.194:41095)
0808 22:27:08.646487 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:08.646598 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":87,"threads_started":1}
W20260808 22:27:08.647130 17620 heartbeater.cc:644] Failed to heartbeat to 127.5.11.254:39325 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
W20260808 22:27:08.647473 17390 heartbeater.cc:644] Failed to heartbeat to 127.5.11.254:39325 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
I20260808 22:27:08.647992 17930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:08.647500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.5.11.195:48135)
0808 22:27:08.647787 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:08.647873 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":74,"threads_started":1}
W20260808 22:27:08.648311 17833 heartbeater.cc:644] Failed to heartbeat to 127.5.11.254:39325 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
I20260808 22:27:09.157737 17931 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b62887ed9ca74e17a55fd6afeadb4cf6)
I20260808 22:27:09.157949 17931 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: false } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: false } }
I20260808 22:27:09.158340 17932 raft_consensus.cc:482] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b62887ed9ca74e17a55fd6afeadb4cf6)
I20260808 22:27:09.158545 17932 raft_consensus.cc:504] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: false } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: false } }
I20260808 22:27:09.159153 17931 leader_election.cc:288] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b62887ed9ca74e17a55fd6afeadb4cf6 (127.5.11.254:39325), 616fc1115cca4edc9017bcb25fc8f143 (127.5.11.253:37661)
I20260808 22:27:09.159775 17935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:09.159238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.0.0.1:51856)
0808 22:27:09.159554 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:09.159642 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":71,"threads_started":1}
W20260808 22:27:09.159960 17073 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
I20260808 22:27:09.160115 17937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:09.159497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.0.0.1:51864)
0808 22:27:09.159778 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:09.159957 (+ 179us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":58,"threads_started":1}
I20260808 22:27:09.160277 17932 leader_election.cc:288] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b62887ed9ca74e17a55fd6afeadb4cf6 (127.5.11.254:39325), f2de4baf12b64a80b7ee6544655fdb81 (127.5.11.252:38217)
W20260808 22:27:09.160364 16800 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
I20260808 22:27:09.161064 17935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:09.160778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.0.0.1:51878)
0808 22:27:09.160929 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:09.160986 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260808 22:27:09.161252 17073 leader_election.cc:334] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b62887ed9ca74e17a55fd6afeadb4cf6 (127.5.11.254:39325): Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
I20260808 22:27:09.161342 17937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:09.161099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.0.0.1:51888)
0808 22:27:09.161223 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:09.161279 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 22:27:09.161432 16800 leader_election.cc:334] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b62887ed9ca74e17a55fd6afeadb4cf6 (127.5.11.254:39325): Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
I20260808 22:27:09.164954 17939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:09.159967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:38217 (local address 127.0.0.1:47506)
0808 22:27:09.160162 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:09.160179 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:09.160270 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:09.161054 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:09.161058 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:09.161080 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:27:09.161224 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:09.161232 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:09.162300 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:09.162303 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:09.163605 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:09.163616 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:09.164478 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:09.164482 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:09.164619 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:09.164645 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 22:27:09.164726 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 22:27:09.164793 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":21,"thread_start_us":71,"threads_started":1}
I20260808 22:27:09.164954 17936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:09.159388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.253:37661 (local address 127.0.0.1:54284)
0808 22:27:09.159679 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:09.159696 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:09.159829 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:09.161244 (+ 1415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:09.161247 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:09.161268 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:27:09.161364 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:09.161370 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:09.162405 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:09.162409 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:09.163605 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:09.163616 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:09.164478 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:09.164482 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:09.164619 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:09.164645 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 22:27:09.164726 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 22:27:09.164793 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":19,"thread_start_us":71,"threads_started":1}
I20260808 22:27:09.164973 17944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:09.160314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47506 (local address 127.5.11.252:38217)
0808 22:27:09.160795 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:09.160805 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:09.160818 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:27:09.160863 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:09.160866 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:09.160907 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:09.161024 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:27:09.161347 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:09.162184 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:09.163755 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:09.164354 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:09.164465 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:09.164594 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:09.164755 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 22:27:09.164828 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":417,"thread_start_us":96,"threads_started":1}
I20260808 22:27:09.164983 17938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:09.159467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54284 (local address 127.5.11.253:37661)
0808 22:27:09.160655 (+ 1188us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:09.160660 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:09.160674 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:27:09.160719 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:09.160722 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:09.160767 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:09.161456 (+ 689us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:27:09.161558 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:09.162297 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:09.163736 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:09.164354 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:09.164465 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:09.164592 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:09.164755 (+ 163us) server_negotiation.cc:299] Negotiation successful
0808 22:27:09.164828 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":1113,"thread_start_us":102,"threads_started":1}
I20260808 22:27:09.165659 17114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "616fc1115cca4edc9017bcb25fc8f143" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f2de4baf12b64a80b7ee6544655fdb81" is_pre_election: true
I20260808 22:27:09.165659 16831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f2de4baf12b64a80b7ee6544655fdb81" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "616fc1115cca4edc9017bcb25fc8f143" is_pre_election: true
I20260808 22:27:09.165859 17114 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 616fc1115cca4edc9017bcb25fc8f143 in term 1.
I20260808 22:27:09.165859 16831 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2de4baf12b64a80b7ee6544655fdb81 in term 1.
I20260808 22:27:09.166183 17071 leader_election.cc:302] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 616fc1115cca4edc9017bcb25fc8f143, f2de4baf12b64a80b7ee6544655fdb81; no voters: b62887ed9ca74e17a55fd6afeadb4cf6
I20260808 22:27:09.166224 16802 leader_election.cc:302] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 616fc1115cca4edc9017bcb25fc8f143, f2de4baf12b64a80b7ee6544655fdb81; no voters: b62887ed9ca74e17a55fd6afeadb4cf6
I20260808 22:27:09.166448 17932 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 22:27:09.166442 17931 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260808 22:27:09.166517 17932 raft_consensus.cc:482] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b62887ed9ca74e17a55fd6afeadb4cf6)
I20260808 22:27:09.166528 17931 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b62887ed9ca74e17a55fd6afeadb4cf6)
I20260808 22:27:09.166561 17932 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [term 1 FOLLOWER]: Advancing to term 2
I20260808 22:27:09.166579 17931 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [term 1 FOLLOWER]: Advancing to term 2
I20260808 22:27:09.167582 17932 raft_consensus.cc:504] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: false } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: false } }
I20260808 22:27:09.167590 17931 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: false } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: false } }
I20260808 22:27:09.167979 17932 leader_election.cc:288] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [CANDIDATE]: Term 2 election: Requested vote from peers b62887ed9ca74e17a55fd6afeadb4cf6 (127.5.11.254:39325), f2de4baf12b64a80b7ee6544655fdb81 (127.5.11.252:38217)
I20260808 22:27:09.167982 17931 leader_election.cc:288] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [CANDIDATE]: Term 2 election: Requested vote from peers b62887ed9ca74e17a55fd6afeadb4cf6 (127.5.11.254:39325), 616fc1115cca4edc9017bcb25fc8f143 (127.5.11.253:37661)
I20260808 22:27:09.168298 17939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:09.168087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.0.0.1:51892)
0808 22:27:09.168183 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:09.168235 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 22:27:09.168393 16831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f2de4baf12b64a80b7ee6544655fdb81" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "616fc1115cca4edc9017bcb25fc8f143"
I20260808 22:27:09.168578 16831 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate f2de4baf12b64a80b7ee6544655fdb81 in current term 2: Already voted for candidate 616fc1115cca4edc9017bcb25fc8f143 in this term.
I20260808 22:27:09.168650 17114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "616fc1115cca4edc9017bcb25fc8f143" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f2de4baf12b64a80b7ee6544655fdb81"
I20260808 22:27:09.168761 17936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:09.168373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.0.0.1:51906)
0808 22:27:09.168645 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:09.168697 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208}
I20260808 22:27:09.168810 17114 raft_consensus.cc:2379] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 616fc1115cca4edc9017bcb25fc8f143 in current term 2: Already voted for candidate f2de4baf12b64a80b7ee6544655fdb81 in this term.
I20260808 22:27:09.169351 17939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:09.169097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.0.0.1:51920)
0808 22:27:09.169254 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:09.169302 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260808 22:27:09.169365 17936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:09.169152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.0.0.1:51922)
0808 22:27:09.169270 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:09.169318 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260808 22:27:09.169468 17073 leader_election.cc:334] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b62887ed9ca74e17a55fd6afeadb4cf6 (127.5.11.254:39325): Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
W20260808 22:27:09.169459 16800 leader_election.cc:334] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b62887ed9ca74e17a55fd6afeadb4cf6 (127.5.11.254:39325): Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
I20260808 22:27:09.169534 17073 leader_election.cc:302] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f2de4baf12b64a80b7ee6544655fdb81; no voters: 616fc1115cca4edc9017bcb25fc8f143, b62887ed9ca74e17a55fd6afeadb4cf6
I20260808 22:27:09.169584 16800 leader_election.cc:302] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 616fc1115cca4edc9017bcb25fc8f143; no voters: b62887ed9ca74e17a55fd6afeadb4cf6, f2de4baf12b64a80b7ee6544655fdb81
I20260808 22:27:09.169663 17931 raft_consensus.cc:2726] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 22:27:09.169832 17932 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260808 22:27:09.648645 17948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:09.647905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.5.11.194:36821)
0808 22:27:09.648286 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:09.648359 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":90,"threads_started":1}
I20260808 22:27:09.648576 17949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:09.648081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.5.11.193:35337)
0808 22:27:09.648379 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:09.648451 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":66,"threads_started":1}
I20260808 22:27:09.649684 17950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:09.648824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.5.11.195:56999)
0808 22:27:09.649165 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:09.649570 (+ 405us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":75,"threads_started":1}
I20260808 22:27:10.650679 17954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:10.649965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.5.11.193:59107)
0808 22:27:10.650246 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:10.650545 (+ 299us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":75,"threads_started":1}
I20260808 22:27:10.650794 17953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:10.649563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.5.11.194:55351)
0808 22:27:10.649895 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:10.649969 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":88,"threads_started":1}
I20260808 22:27:10.650930 17955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:10.650328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.5.11.195:37605)
0808 22:27:10.650760 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:10.650829 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":63,"threads_started":1}
I20260808 22:27:10.683343 17956 raft_consensus.cc:482] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:27:10.683476 17956 raft_consensus.cc:504] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: false } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: false } }
I20260808 22:27:10.683917 17956 leader_election.cc:288] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b62887ed9ca74e17a55fd6afeadb4cf6 (127.5.11.254:39325), f2de4baf12b64a80b7ee6544655fdb81 (127.5.11.252:38217)
I20260808 22:27:10.684319 17114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "616fc1115cca4edc9017bcb25fc8f143" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f2de4baf12b64a80b7ee6544655fdb81" is_pre_election: true
I20260808 22:27:10.684442 17114 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 616fc1115cca4edc9017bcb25fc8f143 in term 2.
I20260808 22:27:10.684502 17957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:10.684073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.0.0.1:52180)
0808 22:27:10.684322 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:10.684380 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":117,"threads_started":1}
I20260808 22:27:10.684788 16802 leader_election.cc:302] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 616fc1115cca4edc9017bcb25fc8f143, f2de4baf12b64a80b7ee6544655fdb81; no voters:
I20260808 22:27:10.685011 17956 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260808 22:27:10.685082 17956 raft_consensus.cc:482] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:27:10.685137 17956 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [term 2 FOLLOWER]: Advancing to term 3
I20260808 22:27:10.685374 17957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:10.685164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.0.0.1:52196)
0808 22:27:10.685270 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:10.685323 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 22:27:10.685479 16800 leader_election.cc:334] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b62887ed9ca74e17a55fd6afeadb4cf6 (127.5.11.254:39325): Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
I20260808 22:27:10.686378 17956 raft_consensus.cc:504] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: false } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: false } }
I20260808 22:27:10.686713 17956 leader_election.cc:288] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [CANDIDATE]: Term 3 election: Requested vote from peers b62887ed9ca74e17a55fd6afeadb4cf6 (127.5.11.254:39325), f2de4baf12b64a80b7ee6544655fdb81 (127.5.11.252:38217)
I20260808 22:27:10.687078 17957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:10.686885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.0.0.1:52204)
0808 22:27:10.686966 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:10.687022 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260808 22:27:10.687072 17114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "616fc1115cca4edc9017bcb25fc8f143" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "f2de4baf12b64a80b7ee6544655fdb81"
I20260808 22:27:10.687194 17114 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [term 2 FOLLOWER]: Advancing to term 3
I20260808 22:27:10.687815 17957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:10.687583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.0.0.1:52216)
0808 22:27:10.687690 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:10.687757 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 22:27:10.687965 16800 leader_election.cc:334] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer b62887ed9ca74e17a55fd6afeadb4cf6 (127.5.11.254:39325): Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
I20260808 22:27:10.688377 17114 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 616fc1115cca4edc9017bcb25fc8f143 in term 3.
I20260808 22:27:10.688658 16802 leader_election.cc:302] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 616fc1115cca4edc9017bcb25fc8f143, f2de4baf12b64a80b7ee6544655fdb81; no voters: b62887ed9ca74e17a55fd6afeadb4cf6
I20260808 22:27:10.688835 17956 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 22:27:10.689029 17956 raft_consensus.cc:686] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [term 3 LEADER]: Becoming Leader. State: Replica: 616fc1115cca4edc9017bcb25fc8f143, State: Running, Role: LEADER
I20260808 22:27:10.689303 17956 consensus_queue.cc:237] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: false } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: false } }
I20260808 22:27:10.690354 17960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 616fc1115cca4edc9017bcb25fc8f143. Latest consensus state: current_term: 3 leader_uuid: "616fc1115cca4edc9017bcb25fc8f143" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: false } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: false } } }
I20260808 22:27:10.690490 17960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [sys.catalog]: This master's current role is: LEADER
I20260808 22:27:10.690788 17964 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:27:10.693419 17964 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:27:10.693816 17964 catalog_manager.cc:1259] Loaded cluster ID: e6cced0e5a9e49409b85ee0fb2c9be1c
I20260808 22:27:10.693863 17964 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:27:10.694427 17964 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:27:10.694823 17964 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143: Loaded TSK: 0
I20260808 22:27:10.695518 17964 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:27:10.740617 16818 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54280:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:27:10.743976 16818 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:27:10.747143 17957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:10.746886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.0.0.1:52244)
0808 22:27:10.747007 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:10.747069 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 22:27:10.747246 17114 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [term 3 FOLLOWER]: Refusing update from remote peer 616fc1115cca4edc9017bcb25fc8f143: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260808 22:27:10.747783 17960 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260808 22:27:10.747926 17957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:10.747677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.0.0.1:52258)
0808 22:27:10.747813 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:10.747866 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 22:27:10.748114 16800 consensus_peers.cc:487] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 -> Peer b62887ed9ca74e17a55fd6afeadb4cf6 (127.5.11.254:39325): Couldn't send request to peer b62887ed9ca74e17a55fd6afeadb4cf6. Status: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260808 22:27:10.749222 17968 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 616fc1115cca4edc9017bcb25fc8f143. Latest consensus state: current_term: 3 leader_uuid: "616fc1115cca4edc9017bcb25fc8f143" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: false } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: false } } }
I20260808 22:27:10.749341 17968 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:27:10.750326 17956 sys_catalog.cc:455] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "616fc1115cca4edc9017bcb25fc8f143" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: false } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: false } } }
I20260808 22:27:10.750445 17956 sys_catalog.cc:458] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143 [sys.catalog]: This master's current role is: LEADER
I20260808 22:27:10.751549 17968 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "616fc1115cca4edc9017bcb25fc8f143" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b62887ed9ca74e17a55fd6afeadb4cf6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39325 } } peers { permanent_uuid: "616fc1115cca4edc9017bcb25fc8f143" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 37661 } attrs { promote: false } } peers { permanent_uuid: "f2de4baf12b64a80b7ee6544655fdb81" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 38217 } attrs { promote: false } } }
I20260808 22:27:10.751680 17968 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2de4baf12b64a80b7ee6544655fdb81 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:27:10.761166 17982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:10.756341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:45457 (local address 127.0.0.1:55088)
0808 22:27:10.756898 (+ 557us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:10.756915 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:10.757014 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:10.757733 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:10.757736 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:10.757752 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:27:10.757827 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:10.757832 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:10.758672 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:10.758677 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:10.759848 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:10.759860 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:10.760731 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:10.760734 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:10.760827 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:10.760856 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 22:27:10.760937 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 22:27:10.761032 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":500,"thread_start_us":97,"threads_started":1}
I20260808 22:27:10.761194 17980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:10.756440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55088 (local address 127.5.11.195:45457)
0808 22:27:10.757507 (+ 1067us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:10.757512 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:10.757526 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:27:10.757574 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:10.757577 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:10.757620 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:10.757713 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:27:10.757985 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:10.758556 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:10.759998 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:10.760611 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:10.760718 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:10.760795 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:10.760970 (+ 175us) server_negotiation.cc:299] Negotiation successful
0808 22:27:10.761051 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":951,"thread_start_us":64,"threads_started":1}
I20260808 22:27:10.761442 17957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:10.755975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:39203 (local address 127.0.0.1:42682)
0808 22:27:10.756134 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:10.756148 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:10.756238 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:10.757893 (+ 1655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:10.757899 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:10.757920 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:27:10.758002 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:10.758008 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:10.759047 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:10.759050 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:10.760273 (+ 1223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:10.760283 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:10.761079 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:10.761083 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:10.761165 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:10.761197 (+ 32us) client_negotiation.cc:769] Sending connection context
0808 22:27:10.761255 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 22:27:10.761340 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"mutex_wait_us":88}
I20260808 22:27:10.761801 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:10.756616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:43785 (local address 127.0.0.1:42414)
0808 22:27:10.757178 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:10.757190 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:10.757268 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:10.757531 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:10.757535 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:10.757556 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:27:10.757674 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:10.757683 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:10.759034 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:10.759038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:10.760316 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:10.760325 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:10.761388 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:10.761396 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:10.761462 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:10.761481 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 22:27:10.761531 (+ 50us) client_negotiation.cc:240] Negotiation successful
0808 22:27:10.761630 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"mutex_wait_us":68,"thread_start_us":95,"threads_started":1}
I20260808 22:27:10.761911 17978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:10.756084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42682 (local address 127.5.11.193:39203)
0808 22:27:10.757047 (+ 963us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:10.757054 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:10.757071 (+ 17us) server_negotiation.cc:407] Connection header received
0808 22:27:10.757127 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:10.757130 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:10.757184 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:10.757871 (+ 687us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:27:10.758123 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:10.758905 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:10.760413 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:10.760953 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:10.761561 (+ 608us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:10.761642 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:10.761680 (+ 38us) server_negotiation.cc:299] Negotiation successful
0808 22:27:10.761756 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":882,"thread_start_us":93,"threads_started":1}
I20260808 22:27:10.762302 17720 tablet_service.cc:1467] Processing CreateTablet for tablet 48e31eecc32247d99ff30042222c1b05 (DEFAULT_TABLE table=test-table [id=3f5187d663d84d71bdc63500ce743eb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:27:10.762504 17276 tablet_service.cc:1467] Processing CreateTablet for tablet 48e31eecc32247d99ff30042222c1b05 (DEFAULT_TABLE table=test-table [id=3f5187d663d84d71bdc63500ce743eb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:27:10.762655 17720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48e31eecc32247d99ff30042222c1b05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:27:10.762830 17276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48e31eecc32247d99ff30042222c1b05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:27:10.764184 17983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:10.756731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42414 (local address 127.5.11.194:43785)
0808 22:27:10.757217 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:10.757222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:10.757244 (+ 22us) server_negotiation.cc:407] Connection header received
0808 22:27:10.757363 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:10.757366 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:10.757419 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:10.757491 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:27:10.757822 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:10.758905 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:10.760456 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:10.761105 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:10.761215 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:10.763898 (+ 2683us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:10.763942 (+ 44us) server_negotiation.cc:299] Negotiation successful
0808 22:27:10.764045 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":73,"threads_started":1}
I20260808 22:27:10.765039 17985 tablet_bootstrap.cc:492] T 48e31eecc32247d99ff30042222c1b05 P feed9713203a46bfb1c3a49ba5cecd66: Bootstrap starting.
I20260808 22:27:10.765132 17986 tablet_bootstrap.cc:492] T 48e31eecc32247d99ff30042222c1b05 P f43f8e31fb01495fabaefe56425dc079: Bootstrap starting.
I20260808 22:27:10.765441 17514 tablet_service.cc:1467] Processing CreateTablet for tablet 48e31eecc32247d99ff30042222c1b05 (DEFAULT_TABLE table=test-table [id=3f5187d663d84d71bdc63500ce743eb7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:27:10.765720 17514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 48e31eecc32247d99ff30042222c1b05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:27:10.766100 17986 tablet_bootstrap.cc:654] T 48e31eecc32247d99ff30042222c1b05 P f43f8e31fb01495fabaefe56425dc079: Neither blocks nor log segments found. Creating new log.
I20260808 22:27:10.767467 17986 tablet_bootstrap.cc:492] T 48e31eecc32247d99ff30042222c1b05 P f43f8e31fb01495fabaefe56425dc079: No bootstrap required, opened a new log
I20260808 22:27:10.767544 17986 ts_tablet_manager.cc:1397] T 48e31eecc32247d99ff30042222c1b05 P f43f8e31fb01495fabaefe56425dc079: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 22:27:10.768072 17986 raft_consensus.cc:348] T 48e31eecc32247d99ff30042222c1b05 P f43f8e31fb01495fabaefe56425dc079 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f43f8e31fb01495fabaefe56425dc079" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39203 } } peers { permanent_uuid: "feed9713203a46bfb1c3a49ba5cecd66" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 45457 } } peers { permanent_uuid: "bc97c1ed9ee6470b99e8b230c5c84b61" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 43785 } }
I20260808 22:27:10.768198 17986 raft_consensus.cc:374] T 48e31eecc32247d99ff30042222c1b05 P f43f8e31fb01495fabaefe56425dc079 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:27:10.768258 17986 raft_consensus.cc:729] T 48e31eecc32247d99ff30042222c1b05 P f43f8e31fb01495fabaefe56425dc079 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f43f8e31fb01495fabaefe56425dc079, State: Initialized, Role: FOLLOWER
I20260808 22:27:10.768400 17986 consensus_queue.cc:260] T 48e31eecc32247d99ff30042222c1b05 P f43f8e31fb01495fabaefe56425dc079 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f43f8e31fb01495fabaefe56425dc079" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39203 } } peers { permanent_uuid: "feed9713203a46bfb1c3a49ba5cecd66" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 45457 } } peers { permanent_uuid: "bc97c1ed9ee6470b99e8b230c5c84b61" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 43785 } }
I20260808 22:27:10.768805 17985 tablet_bootstrap.cc:654] T 48e31eecc32247d99ff30042222c1b05 P feed9713203a46bfb1c3a49ba5cecd66: Neither blocks nor log segments found. Creating new log.
I20260808 22:27:10.768818 17986 ts_tablet_manager.cc:1428] T 48e31eecc32247d99ff30042222c1b05 P f43f8e31fb01495fabaefe56425dc079: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 22:27:10.769420 17954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:10.769069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.5.11.193:47563)
0808 22:27:10.769294 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:10.769352 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
I20260808 22:27:10.769773 17989 tablet_bootstrap.cc:492] T 48e31eecc32247d99ff30042222c1b05 P bc97c1ed9ee6470b99e8b230c5c84b61: Bootstrap starting.
I20260808 22:27:10.770092 17954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:10.769895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.5.11.193:35693)
0808 22:27:10.769987 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:10.770033 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 22:27:10.770726 17989 tablet_bootstrap.cc:654] T 48e31eecc32247d99ff30042222c1b05 P bc97c1ed9ee6470b99e8b230c5c84b61: Neither blocks nor log segments found. Creating new log.
I20260808 22:27:10.771189 17391 heartbeater.cc:503] Master 127.5.11.253:37661 was elected leader, sending a full tablet report...
I20260808 22:27:10.772156 17989 tablet_bootstrap.cc:492] T 48e31eecc32247d99ff30042222c1b05 P bc97c1ed9ee6470b99e8b230c5c84b61: No bootstrap required, opened a new log
I20260808 22:27:10.772253 17989 ts_tablet_manager.cc:1397] T 48e31eecc32247d99ff30042222c1b05 P bc97c1ed9ee6470b99e8b230c5c84b61: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:27:10.772732 17989 raft_consensus.cc:348] T 48e31eecc32247d99ff30042222c1b05 P bc97c1ed9ee6470b99e8b230c5c84b61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f43f8e31fb01495fabaefe56425dc079" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39203 } } peers { permanent_uuid: "feed9713203a46bfb1c3a49ba5cecd66" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 45457 } } peers { permanent_uuid: "bc97c1ed9ee6470b99e8b230c5c84b61" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 43785 } }
I20260808 22:27:10.772855 17989 raft_consensus.cc:374] T 48e31eecc32247d99ff30042222c1b05 P bc97c1ed9ee6470b99e8b230c5c84b61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:27:10.772907 17989 raft_consensus.cc:729] T 48e31eecc32247d99ff30042222c1b05 P bc97c1ed9ee6470b99e8b230c5c84b61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc97c1ed9ee6470b99e8b230c5c84b61, State: Initialized, Role: FOLLOWER
I20260808 22:27:10.773065 17989 consensus_queue.cc:260] T 48e31eecc32247d99ff30042222c1b05 P bc97c1ed9ee6470b99e8b230c5c84b61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f43f8e31fb01495fabaefe56425dc079" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39203 } } peers { permanent_uuid: "feed9713203a46bfb1c3a49ba5cecd66" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 45457 } } peers { permanent_uuid: "bc97c1ed9ee6470b99e8b230c5c84b61" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 43785 } }
I20260808 22:27:10.773474 17989 ts_tablet_manager.cc:1428] T 48e31eecc32247d99ff30042222c1b05 P bc97c1ed9ee6470b99e8b230c5c84b61: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260808 22:27:10.773574 17985 tablet_bootstrap.cc:492] T 48e31eecc32247d99ff30042222c1b05 P feed9713203a46bfb1c3a49ba5cecd66: No bootstrap required, opened a new log
I20260808 22:27:10.773720 17985 ts_tablet_manager.cc:1397] T 48e31eecc32247d99ff30042222c1b05 P feed9713203a46bfb1c3a49ba5cecd66: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260808 22:27:10.774247 17985 raft_consensus.cc:348] T 48e31eecc32247d99ff30042222c1b05 P feed9713203a46bfb1c3a49ba5cecd66 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f43f8e31fb01495fabaefe56425dc079" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39203 } } peers { permanent_uuid: "feed9713203a46bfb1c3a49ba5cecd66" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 45457 } } peers { permanent_uuid: "bc97c1ed9ee6470b99e8b230c5c84b61" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 43785 } }
I20260808 22:27:10.774394 17985 raft_consensus.cc:374] T 48e31eecc32247d99ff30042222c1b05 P feed9713203a46bfb1c3a49ba5cecd66 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:27:10.774436 17953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:10.774166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.5.11.194:35351)
0808 22:27:10.774316 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:10.774366 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260808 22:27:10.774459 17985 raft_consensus.cc:729] T 48e31eecc32247d99ff30042222c1b05 P feed9713203a46bfb1c3a49ba5cecd66 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: feed9713203a46bfb1c3a49ba5cecd66, State: Initialized, Role: FOLLOWER
I20260808 22:27:10.774598 17985 consensus_queue.cc:260] T 48e31eecc32247d99ff30042222c1b05 P feed9713203a46bfb1c3a49ba5cecd66 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f43f8e31fb01495fabaefe56425dc079" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39203 } } peers { permanent_uuid: "feed9713203a46bfb1c3a49ba5cecd66" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 45457 } } peers { permanent_uuid: "bc97c1ed9ee6470b99e8b230c5c84b61" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 43785 } }
I20260808 22:27:10.775044 17985 ts_tablet_manager.cc:1428] T 48e31eecc32247d99ff30042222c1b05 P feed9713203a46bfb1c3a49ba5cecd66: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:27:10.775156 17621 heartbeater.cc:503] Master 127.5.11.253:37661 was elected leader, sending a full tablet report...
I20260808 22:27:10.775190 17953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:10.774932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.5.11.194:59941)
0808 22:27:10.775068 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:10.775120 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260808 22:27:10.775950 17955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:10.775686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.5.11.195:44017)
0808 22:27:10.775843 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:10.775895 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260808 22:27:10.776639 17955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:10.776448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.5.11.195:37433)
0808 22:27:10.776536 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:10.776580 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260808 22:27:10.777257 17834 heartbeater.cc:503] Master 127.5.11.253:37661 was elected leader, sending a full tablet report...
I20260808 22:27:10.782943 17990 raft_consensus.cc:482] T 48e31eecc32247d99ff30042222c1b05 P f43f8e31fb01495fabaefe56425dc079 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:27:10.783070 17990 raft_consensus.cc:504] T 48e31eecc32247d99ff30042222c1b05 P f43f8e31fb01495fabaefe56425dc079 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f43f8e31fb01495fabaefe56425dc079" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39203 } } peers { permanent_uuid: "feed9713203a46bfb1c3a49ba5cecd66" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 45457 } } peers { permanent_uuid: "bc97c1ed9ee6470b99e8b230c5c84b61" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 43785 } }
I20260808 22:27:10.784173 17990 leader_election.cc:288] T 48e31eecc32247d99ff30042222c1b05 P f43f8e31fb01495fabaefe56425dc079 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers feed9713203a46bfb1c3a49ba5cecd66 (127.5.11.195:45457), bc97c1ed9ee6470b99e8b230c5c84b61 (127.5.11.194:43785)
I20260808 22:27:10.789316 17980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:10.784450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:57911 (local address 127.5.11.195:45457)
0808 22:27:10.784663 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:10.784667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:10.784682 (+ 15us) server_negotiation.cc:407] Connection header received
0808 22:27:10.784742 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:10.784745 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:10.784791 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:10.784888 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:27:10.785224 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:10.786018 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:10.787939 (+ 1921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:10.788537 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:10.788651 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:10.788699 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:10.789144 (+ 445us) server_negotiation.cc:299] Negotiation successful
0808 22:27:10.789212 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153}
I20260808 22:27:10.789317 17983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:10.784565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:59095 (local address 127.5.11.194:43785)
0808 22:27:10.784663 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:10.784667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:10.784948 (+ 281us) server_negotiation.cc:407] Connection header received
0808 22:27:10.785036 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:10.785038 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:10.785070 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:10.785126 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:27:10.785378 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:10.786110 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:10.787939 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:10.788531 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:10.788651 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:10.788699 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:10.789144 (+ 445us) server_negotiation.cc:299] Negotiation successful
0808 22:27:10.789212 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 22:27:10.789366 17954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:10.784356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:45457 (local address 127.5.11.193:57911)
0808 22:27:10.784488 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:10.784503 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:10.784587 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:10.784924 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:10.784927 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:10.784948 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:27:10.785078 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:10.785085 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:10.786121 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:10.786124 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:10.787814 (+ 1690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:10.787825 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:10.788679 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:10.788683 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:10.788941 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:10.789034 (+ 93us) client_negotiation.cc:769] Sending connection context
0808 22:27:10.789114 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 22:27:10.789202 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":268}
I20260808 22:27:10.789379 17997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:10.784373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:43785 (local address 127.5.11.193:59095)
0808 22:27:10.784839 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:10.784850 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:10.784959 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:10.785156 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:10.785161 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:10.785184 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:27:10.785282 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:10.785286 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:10.786209 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:10.786213 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:10.787815 (+ 1602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:10.787825 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:10.788679 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:10.788683 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:10.788937 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:10.789034 (+ 97us) client_negotiation.cc:769] Sending connection context
0808 22:27:10.789114 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 22:27:10.789202 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"mutex_wait_us":289,"thread_start_us":101,"threads_started":1}
I20260808 22:27:10.789873 17549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48e31eecc32247d99ff30042222c1b05" candidate_uuid: "f43f8e31fb01495fabaefe56425dc079" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc97c1ed9ee6470b99e8b230c5c84b61" is_pre_election: true
I20260808 22:27:10.790033 17549 raft_consensus.cc:2454] T 48e31eecc32247d99ff30042222c1b05 P bc97c1ed9ee6470b99e8b230c5c84b61 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f43f8e31fb01495fabaefe56425dc079 in term 0.
I20260808 22:27:10.789990 17760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48e31eecc32247d99ff30042222c1b05" candidate_uuid: "f43f8e31fb01495fabaefe56425dc079" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "feed9713203a46bfb1c3a49ba5cecd66" is_pre_election: true
I20260808 22:27:10.790243 17760 raft_consensus.cc:2454] T 48e31eecc32247d99ff30042222c1b05 P feed9713203a46bfb1c3a49ba5cecd66 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f43f8e31fb01495fabaefe56425dc079 in term 0.
I20260808 22:27:10.790424 17202 leader_election.cc:302] T 48e31eecc32247d99ff30042222c1b05 P f43f8e31fb01495fabaefe56425dc079 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc97c1ed9ee6470b99e8b230c5c84b61, f43f8e31fb01495fabaefe56425dc079; no voters:
I20260808 22:27:10.790720 17990 raft_consensus.cc:2781] T 48e31eecc32247d99ff30042222c1b05 P f43f8e31fb01495fabaefe56425dc079 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:27:10.790786 17990 raft_consensus.cc:482] T 48e31eecc32247d99ff30042222c1b05 P f43f8e31fb01495fabaefe56425dc079 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:27:10.790824 17990 raft_consensus.cc:3037] T 48e31eecc32247d99ff30042222c1b05 P f43f8e31fb01495fabaefe56425dc079 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:27:10.791827 17990 raft_consensus.cc:504] T 48e31eecc32247d99ff30042222c1b05 P f43f8e31fb01495fabaefe56425dc079 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f43f8e31fb01495fabaefe56425dc079" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39203 } } peers { permanent_uuid: "feed9713203a46bfb1c3a49ba5cecd66" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 45457 } } peers { permanent_uuid: "bc97c1ed9ee6470b99e8b230c5c84b61" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 43785 } }
I20260808 22:27:10.792198 17990 leader_election.cc:288] T 48e31eecc32247d99ff30042222c1b05 P f43f8e31fb01495fabaefe56425dc079 [CANDIDATE]: Term 1 election: Requested vote from peers feed9713203a46bfb1c3a49ba5cecd66 (127.5.11.195:45457), bc97c1ed9ee6470b99e8b230c5c84b61 (127.5.11.194:43785)
I20260808 22:27:10.792523 17760 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48e31eecc32247d99ff30042222c1b05" candidate_uuid: "f43f8e31fb01495fabaefe56425dc079" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "feed9713203a46bfb1c3a49ba5cecd66"
I20260808 22:27:10.792536 17549 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "48e31eecc32247d99ff30042222c1b05" candidate_uuid: "f43f8e31fb01495fabaefe56425dc079" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc97c1ed9ee6470b99e8b230c5c84b61"
I20260808 22:27:10.792656 17760 raft_consensus.cc:3037] T 48e31eecc32247d99ff30042222c1b05 P feed9713203a46bfb1c3a49ba5cecd66 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:27:10.792665 17549 raft_consensus.cc:3037] T 48e31eecc32247d99ff30042222c1b05 P bc97c1ed9ee6470b99e8b230c5c84b61 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:27:10.793471 17549 raft_consensus.cc:2454] T 48e31eecc32247d99ff30042222c1b05 P bc97c1ed9ee6470b99e8b230c5c84b61 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f43f8e31fb01495fabaefe56425dc079 in term 1.
I20260808 22:27:10.793571 17760 raft_consensus.cc:2454] T 48e31eecc32247d99ff30042222c1b05 P feed9713203a46bfb1c3a49ba5cecd66 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f43f8e31fb01495fabaefe56425dc079 in term 1.
I20260808 22:27:10.793751 17202 leader_election.cc:302] T 48e31eecc32247d99ff30042222c1b05 P f43f8e31fb01495fabaefe56425dc079 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc97c1ed9ee6470b99e8b230c5c84b61, f43f8e31fb01495fabaefe56425dc079; no voters:
I20260808 22:27:10.793936 17990 raft_consensus.cc:2781] T 48e31eecc32247d99ff30042222c1b05 P f43f8e31fb01495fabaefe56425dc079 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:27:10.794144 17990 raft_consensus.cc:686] T 48e31eecc32247d99ff30042222c1b05 P f43f8e31fb01495fabaefe56425dc079 [term 1 LEADER]: Becoming Leader. State: Replica: f43f8e31fb01495fabaefe56425dc079, State: Running, Role: LEADER
I20260808 22:27:10.794395 17990 consensus_queue.cc:237] T 48e31eecc32247d99ff30042222c1b05 P f43f8e31fb01495fabaefe56425dc079 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f43f8e31fb01495fabaefe56425dc079" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39203 } } peers { permanent_uuid: "feed9713203a46bfb1c3a49ba5cecd66" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 45457 } } peers { permanent_uuid: "bc97c1ed9ee6470b99e8b230c5c84b61" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 43785 } }
I20260808 22:27:10.795006 17954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:10.794705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.5.11.193:38179)
0808 22:27:10.794904 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:10.794953 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
I20260808 22:27:10.796667 16818 catalog_manager.cc:5495] T 48e31eecc32247d99ff30042222c1b05 P f43f8e31fb01495fabaefe56425dc079 reported cstate change: term changed from 0 to 1, leader changed from <none> to f43f8e31fb01495fabaefe56425dc079 (127.5.11.193). New cstate: current_term: 1 leader_uuid: "f43f8e31fb01495fabaefe56425dc079" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f43f8e31fb01495fabaefe56425dc079" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39203 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "feed9713203a46bfb1c3a49ba5cecd66" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 45457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc97c1ed9ee6470b99e8b230c5c84b61" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 43785 } health_report { overall_health: UNKNOWN } } }
I20260808 22:27:10.811617 17978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:10.808382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42692 (local address 127.5.11.193:39203)
0808 22:27:10.808485 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:10.808497 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:10.808511 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:27:10.808627 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:10.808630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:10.808682 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:10.808769 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:27:10.808987 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:10.809472 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:10.810269 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:10.810469 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:10.810551 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:10.810781 (+ 230us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:27:10.811202 (+ 421us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:27:10.811288 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:10.811372 (+ 84us) server_negotiation.cc:299] Negotiation successful
0808 22:27:10.811422 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260808 22:27:10.814463 17549 raft_consensus.cc:1261] T 48e31eecc32247d99ff30042222c1b05 P bc97c1ed9ee6470b99e8b230c5c84b61 [term 1 FOLLOWER]: Refusing update from remote peer f43f8e31fb01495fabaefe56425dc079: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:27:10.814800 17760 raft_consensus.cc:1261] T 48e31eecc32247d99ff30042222c1b05 P feed9713203a46bfb1c3a49ba5cecd66 [term 1 FOLLOWER]: Refusing update from remote peer f43f8e31fb01495fabaefe56425dc079: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:27:10.815073 17998 consensus_queue.cc:1035] T 48e31eecc32247d99ff30042222c1b05 P f43f8e31fb01495fabaefe56425dc079 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc97c1ed9ee6470b99e8b230c5c84b61" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 43785 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:27:10.815337 17990 consensus_queue.cc:1035] T 48e31eecc32247d99ff30042222c1b05 P f43f8e31fb01495fabaefe56425dc079 [LEADER]: Connected to new peer: Peer: permanent_uuid: "feed9713203a46bfb1c3a49ba5cecd66" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 45457 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:27:10.818007 17955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:10.817753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.5.11.195:45911)
0808 22:27:10.817873 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:10.817933 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260808 22:27:10.819181 17953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:10.817161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.5.11.194:45525)
0808 22:27:10.819041 (+ 1880us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:10.819100 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1808}
I20260808 22:27:10.820195 18001 mvcc.cc:205] Tried to move back new op lower bound from 7316390014211256320 to 7316390014136348672. Current Snapshot: MvccSnapshot[applied={T|T < 7316390014211256320}]
I20260808 22:27:10.821610 17955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:10.821350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.5.11.195:36967)
0808 22:27:10.821487 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:10.821538 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260808 22:27:10.822225 17954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:10.821872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.5.11.193:47235)
0808 22:27:10.822100 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:10.822155 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161}
I20260808 22:27:10.823853 17954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:10.823618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.5.11.193:55471)
0808 22:27:10.823710 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:10.823793 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 22:27:10.825063 17953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:10.824724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39325 (local address 127.5.11.194:57197)
0808 22:27:10.824943 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:10.825002 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:39325: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147}
I20260808 22:27:10.829064 16818 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:27:10.829191 16818 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:27:10.832819 16818 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 616fc1115cca4edc9017bcb25fc8f143: Sending DeleteTablet for 3 replicas of tablet 48e31eecc32247d99ff30042222c1b05
I20260808 22:27:10.833870 17276 tablet_service.cc:1514] Processing DeleteTablet for tablet 48e31eecc32247d99ff30042222c1b05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:27:10 UTC) from {username='oryx', principal='oryx/127.5.11.253@KRBTEST.COM'} at 127.0.0.1:42682
I20260808 22:27:10.833916 17514 tablet_service.cc:1514] Processing DeleteTablet for tablet 48e31eecc32247d99ff30042222c1b05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:27:10 UTC) from {username='oryx', principal='oryx/127.5.11.253@KRBTEST.COM'} at 127.0.0.1:42414
I20260808 22:27:10.833916 17720 tablet_service.cc:1514] Processing DeleteTablet for tablet 48e31eecc32247d99ff30042222c1b05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:27:10 UTC) from {username='oryx', principal='oryx/127.5.11.253@KRBTEST.COM'} at 127.0.0.1:55088
I20260808 22:27:10.834237 18020 tablet_replica.cc:330] T 48e31eecc32247d99ff30042222c1b05 P f43f8e31fb01495fabaefe56425dc079: stopping tablet replica
I20260808 22:27:10.834368 18020 raft_consensus.cc:2229] T 48e31eecc32247d99ff30042222c1b05 P f43f8e31fb01495fabaefe56425dc079 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:27:10.834657 18020 raft_consensus.cc:2258] T 48e31eecc32247d99ff30042222c1b05 P f43f8e31fb01495fabaefe56425dc079 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:27:10.834884 18022 tablet_replica.cc:330] T 48e31eecc32247d99ff30042222c1b05 P bc97c1ed9ee6470b99e8b230c5c84b61: stopping tablet replica
I20260808 22:27:10.834940 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 17032
I20260808 22:27:10.834998 18022 raft_consensus.cc:2229] T 48e31eecc32247d99ff30042222c1b05 P bc97c1ed9ee6470b99e8b230c5c84b61 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:27:10.835789 18022 raft_consensus.cc:2258] T 48e31eecc32247d99ff30042222c1b05 P bc97c1ed9ee6470b99e8b230c5c84b61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:27:10.841245 18022 ts_tablet_manager.cc:1905] T 48e31eecc32247d99ff30042222c1b05 P bc97c1ed9ee6470b99e8b230c5c84b61: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:27:10.843711 18022 ts_tablet_manager.cc:1918] T 48e31eecc32247d99ff30042222c1b05 P bc97c1ed9ee6470b99e8b230c5c84b61: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:27:10.843820 18022 log.cc:1196] T 48e31eecc32247d99ff30042222c1b05 P bc97c1ed9ee6470b99e8b230c5c84b61: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-1/wal/wals/48e31eecc32247d99ff30042222c1b05
I20260808 22:27:10.844046 18022 ts_tablet_manager.cc:1939] T 48e31eecc32247d99ff30042222c1b05 P bc97c1ed9ee6470b99e8b230c5c84b61: Deleting consensus metadata
I20260808 22:27:10.844807 18023 tablet_replica.cc:330] T 48e31eecc32247d99ff30042222c1b05 P feed9713203a46bfb1c3a49ba5cecd66: stopping tablet replica
I20260808 22:27:10.844821 16800 catalog_manager.cc:4841] TS bc97c1ed9ee6470b99e8b230c5c84b61 (127.5.11.194:43785): tablet 48e31eecc32247d99ff30042222c1b05 (table test-table [id=3f5187d663d84d71bdc63500ce743eb7]) successfully deleted
I20260808 22:27:10.844933 18023 raft_consensus.cc:2229] T 48e31eecc32247d99ff30042222c1b05 P feed9713203a46bfb1c3a49ba5cecd66 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:27:10.845080 18023 raft_consensus.cc:2258] T 48e31eecc32247d99ff30042222c1b05 P feed9713203a46bfb1c3a49ba5cecd66 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:27:10.847498 18023 ts_tablet_manager.cc:1905] T 48e31eecc32247d99ff30042222c1b05 P feed9713203a46bfb1c3a49ba5cecd66: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:27:10.849862 18023 ts_tablet_manager.cc:1918] T 48e31eecc32247d99ff30042222c1b05 P feed9713203a46bfb1c3a49ba5cecd66: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:27:10.849963 18023 log.cc:1196] T 48e31eecc32247d99ff30042222c1b05 P feed9713203a46bfb1c3a49ba5cecd66: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786227991988852-5167-0/minicluster-data/ts-2/wal/wals/48e31eecc32247d99ff30042222c1b05
I20260808 22:27:10.850237 18023 ts_tablet_manager.cc:1939] T 48e31eecc32247d99ff30042222c1b05 P feed9713203a46bfb1c3a49ba5cecd66: Deleting consensus metadata
I20260808 22:27:10.850827 16797 catalog_manager.cc:4841] TS feed9713203a46bfb1c3a49ba5cecd66 (127.5.11.195:45457): tablet 48e31eecc32247d99ff30042222c1b05 (table test-table [id=3f5187d663d84d71bdc63500ce743eb7]) successfully deleted
I20260808 22:27:10.855700 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 17400
I20260808 22:27:10.856146 17979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:10.855812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:39203 (local address 127.0.0.1:42694)
0808 22:27:10.856003 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:10.856057 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.193:39203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
W20260808 22:27:10.861106 16797 catalog_manager.cc:4584] TS f43f8e31fb01495fabaefe56425dc079 (127.5.11.193:39203): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 48e31eecc32247d99ff30042222c1b05: Network error: Client connection negotiation failed: client connection to 127.5.11.193:39203: connect: Connection refused (error 111)
I20260808 22:27:10.868585 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 17627
I20260808 22:27:10.883922 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 16588
I20260808 22:27:10.900871 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 16763
2026-08-08T22:27:10Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7059 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:27:10 dist-test-slave-7f01 krb5kdc[18029](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:27:10 dist-test-slave-7f01 krb5kdc[18029](info): set up 2 sockets
Aug 08 22:27:10 dist-test-slave-7f01 krb5kdc[18029](info): commencing operation
krb5kdc: starting...
W20260808 22:27:12.163223 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.232s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:27:12 dist-test-slave-7f01 krb5kdc[18029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:27:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:27:12Z Disabled control of system clock
WARNING: no policy specified for oryx/127.5.11.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.11.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.11.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.254.keytab.
Entry for principal oryx/127.5.11.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.254.keytab.
I20260808 22:27:12.294134 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:43883
--webserver_interface=127.5.11.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.254.keytab
--principal=oryx/127.5.11.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:36109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:43883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:27:12.407856 18046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:12.408121 18046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:12.408162 18046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:12.412278 18046 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:27:12.412334 18046 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260808 22:27:12.412351 18046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:12.412369 18046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:12.412385 18046 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:27:12.412400 18046 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:27:12.417641 18046 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:36109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:43883
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.254.keytab
--principal=oryx/127.5.11.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:43883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:12.418696 18046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:12.419806 18046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:12.428041 18053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:12.428041 18054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:12.428159 18057 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:12.428483 18046 server_base.cc:1034] running on GCE node
I20260808 22:27:12.429117 18046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:12.430277 18046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:12.431468 18046 hybrid_clock.cc:648] HybridClock initialized: now 1786228032431443 us; error 42 us; skew 500 ppm
Aug 08 22:27:12 dist-test-slave-7f01 krb5kdc[18029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228032, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:27:12.437978 18046 init.cc:377] Logged in from keytab as oryx/127.5.11.254@KRBTEST.COM (short username oryx)
I20260808 22:27:12.438830 18046 webserver.cc:460] Webserver started at http://127.5.11.254:39885/ using document root <none> and password file <none>
I20260808 22:27:12.439380 18046 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:12.439441 18046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:12.439615 18046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:27:12.441705 18046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "258796a613b547bb87a6fa5fe06e6556"
format_stamp: "Formatted at 2026-08-08 22:27:12 on dist-test-slave-7f01"
server_key: "f7be76de6220671a666e8c540a6f4fef"
server_key_iv: "05d21d1bacb528d3015de2a8090cc8bb"
server_key_version: "encryptionkey@0"
I20260808 22:27:12.442241 18046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "258796a613b547bb87a6fa5fe06e6556"
format_stamp: "Formatted at 2026-08-08 22:27:12 on dist-test-slave-7f01"
server_key: "f7be76de6220671a666e8c540a6f4fef"
server_key_iv: "05d21d1bacb528d3015de2a8090cc8bb"
server_key_version: "encryptionkey@0"
I20260808 22:27:12.445959 18046 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 22:27:12.448366 18068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:12.449589 18046 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:27:12.449693 18046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "258796a613b547bb87a6fa5fe06e6556"
format_stamp: "Formatted at 2026-08-08 22:27:12 on dist-test-slave-7f01"
server_key: "f7be76de6220671a666e8c540a6f4fef"
server_key_iv: "05d21d1bacb528d3015de2a8090cc8bb"
server_key_version: "encryptionkey@0"
I20260808 22:27:12.449776 18046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:12.469893 18046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:12.472733 18046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:12.472898 18046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:12.480692 18046 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:43883
I20260808 22:27:12.480732 18169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:43883 every 8 connection(s)
I20260808 22:27:12.481637 18046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:27:12.484542 18171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:27:12.490166 18171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 258796a613b547bb87a6fa5fe06e6556: Bootstrap starting.
I20260808 22:27:12.490984 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 18046
I20260808 22:27:12.491133 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:27:12.491470 5167 external_mini_cluster.cc:1428] Setting key dd945cf4480a4d304c44a67e204565c5
I20260808 22:27:12.493222 18171 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 258796a613b547bb87a6fa5fe06e6556: Neither blocks nor log segments found. Creating new log.
I20260808 22:27:12.493964 18171 log.cc:824] T 00000000000000000000000000000000 P 258796a613b547bb87a6fa5fe06e6556: Log is configured to *not* fsync() on all Append() calls
I20260808 22:27:12.496264 18171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 258796a613b547bb87a6fa5fe06e6556: No bootstrap required, opened a new log
Aug 08 22:27:12 dist-test-slave-7f01 krb5kdc[18029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228032, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM
I20260808 22:27:12.500907 18171 raft_consensus.cc:348] T 00000000000000000000000000000000 P 258796a613b547bb87a6fa5fe06e6556 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "258796a613b547bb87a6fa5fe06e6556" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 43883 } }
I20260808 22:27:12.501143 18171 raft_consensus.cc:374] T 00000000000000000000000000000000 P 258796a613b547bb87a6fa5fe06e6556 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:27:12.501206 18171 raft_consensus.cc:729] T 00000000000000000000000000000000 P 258796a613b547bb87a6fa5fe06e6556 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 258796a613b547bb87a6fa5fe06e6556, State: Initialized, Role: FOLLOWER
I20260808 22:27:12.501700 18171 consensus_queue.cc:260] T 00000000000000000000000000000000 P 258796a613b547bb87a6fa5fe06e6556 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "258796a613b547bb87a6fa5fe06e6556" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 43883 } }
I20260808 22:27:12.501858 18171 raft_consensus.cc:388] T 00000000000000000000000000000000 P 258796a613b547bb87a6fa5fe06e6556 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:27:12.501940 18171 raft_consensus.cc:482] T 00000000000000000000000000000000 P 258796a613b547bb87a6fa5fe06e6556 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:27:12.502055 18171 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 258796a613b547bb87a6fa5fe06e6556 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:27:12.503253 18171 raft_consensus.cc:504] T 00000000000000000000000000000000 P 258796a613b547bb87a6fa5fe06e6556 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "258796a613b547bb87a6fa5fe06e6556" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 43883 } }
I20260808 22:27:12.503664 18171 leader_election.cc:302] T 00000000000000000000000000000000 P 258796a613b547bb87a6fa5fe06e6556 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 258796a613b547bb87a6fa5fe06e6556; no voters:
I20260808 22:27:12.504038 18171 leader_election.cc:288] T 00000000000000000000000000000000 P 258796a613b547bb87a6fa5fe06e6556 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:27:12.504161 18177 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 258796a613b547bb87a6fa5fe06e6556 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:27:12.504456 18177 raft_consensus.cc:686] T 00000000000000000000000000000000 P 258796a613b547bb87a6fa5fe06e6556 [term 1 LEADER]: Becoming Leader. State: Replica: 258796a613b547bb87a6fa5fe06e6556, State: Running, Role: LEADER
I20260808 22:27:12.504829 18177 consensus_queue.cc:237] T 00000000000000000000000000000000 P 258796a613b547bb87a6fa5fe06e6556 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "258796a613b547bb87a6fa5fe06e6556" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 43883 } }
I20260808 22:27:12.505452 18171 sys_catalog.cc:564] T 00000000000000000000000000000000 P 258796a613b547bb87a6fa5fe06e6556 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:27:12.506258 18181 sys_catalog.cc:455] T 00000000000000000000000000000000 P 258796a613b547bb87a6fa5fe06e6556 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 258796a613b547bb87a6fa5fe06e6556. Latest consensus state: current_term: 1 leader_uuid: "258796a613b547bb87a6fa5fe06e6556" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "258796a613b547bb87a6fa5fe06e6556" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 43883 } } }
I20260808 22:27:12.506500 18181 sys_catalog.cc:458] T 00000000000000000000000000000000 P 258796a613b547bb87a6fa5fe06e6556 [sys.catalog]: This master's current role is: LEADER
I20260808 22:27:12.506695 18180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 258796a613b547bb87a6fa5fe06e6556 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "258796a613b547bb87a6fa5fe06e6556" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "258796a613b547bb87a6fa5fe06e6556" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 43883 } } }
I20260808 22:27:12.506887 18180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 258796a613b547bb87a6fa5fe06e6556 [sys.catalog]: This master's current role is: LEADER
I20260808 22:27:12.507265 18194 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:27:12.507414 18175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:12.492669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41418 (local address 127.5.11.254:43883)
0808 22:27:12.493135 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:12.493151 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:12.493188 (+ 37us) server_negotiation.cc:407] Connection header received
0808 22:27:12.493894 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:12.493912 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:12.494302 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:12.494712 (+ 410us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:12.495605 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:12.496699 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:12.497867 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:12.498329 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:12.498446 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:12.501317 (+ 2871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:12.501328 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:12.501341 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:12.501368 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:12.503468 (+ 2100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:12.503969 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:12.503978 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:12.503987 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:12.504147 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:12.504534 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:12.504539 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:12.504544 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:12.505242 (+ 698us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:12.505410 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:12.505930 (+ 520us) server_negotiation.cc:299] Negotiation successful
0808 22:27:12.506267 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":144,"threads_started":1}
I20260808 22:27:12.510571 18194 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:27:12.516189 18194 catalog_manager.cc:1347] Generated new cluster ID: 450a04b81ee140559c9fe31ac653ec54
I20260808 22:27:12.516258 18194 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:27:12.529556 18194 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:27:12.530740 18194 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:27:12.538250 18194 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 258796a613b547bb87a6fa5fe06e6556: Generated new TSK 0
I20260808 22:27:12.539017 18194 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.5.11.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.11.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.11.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.193.keytab.
Entry for principal oryx/127.5.11.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.193.keytab.
I20260808 22:27:12.568941 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.193:0
--local_ip_for_outbound_sockets=127.5.11.193
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:43883
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.193.keytab
--principal=oryx/127.5.11.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:36109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:27:12.682456 18217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:12.682734 18217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:12.682811 18217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:12.686818 18217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:12.686913 18217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:12.686985 18217 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 22:27:12.687065 18217 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.193
I20260808 22:27:12.692209 18217 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:36109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.193.keytab
--principal=oryx/127.5.11.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.11.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.11.254:43883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.193
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:12.693235 18217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:12.694260 18217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:12.704224 18229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:12.704234 18227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:12.704234 18225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:12.704324 18217 server_base.cc:1034] running on GCE node
I20260808 22:27:12.704864 18217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:12.705605 18217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:12.706799 18217 hybrid_clock.cc:648] HybridClock initialized: now 1786228032706786 us; error 34 us; skew 500 ppm
Aug 08 22:27:12 dist-test-slave-7f01 krb5kdc[18029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228032, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:27:12.709697 18217 init.cc:377] Logged in from keytab as oryx/127.5.11.193@KRBTEST.COM (short username oryx)
I20260808 22:27:12.710490 18217 webserver.cc:460] Webserver started at http://127.5.11.193:37865/ using document root <none> and password file <none>
I20260808 22:27:12.711000 18217 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:12.711071 18217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:12.711231 18217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:27:12.713255 18217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-0/data/instance:
uuid: "638feafd83954241a681cb7e2ab8ec64"
format_stamp: "Formatted at 2026-08-08 22:27:12 on dist-test-slave-7f01"
server_key: "b1a107e378d9b003e61d76cfb3539641"
server_key_iv: "9b477372c26b763bf0b9d132541e7117"
server_key_version: "encryptionkey@0"
I20260808 22:27:12.713779 18217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance:
uuid: "638feafd83954241a681cb7e2ab8ec64"
format_stamp: "Formatted at 2026-08-08 22:27:12 on dist-test-slave-7f01"
server_key: "b1a107e378d9b003e61d76cfb3539641"
server_key_iv: "9b477372c26b763bf0b9d132541e7117"
server_key_version: "encryptionkey@0"
I20260808 22:27:12.717482 18217 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 22:27:12.719885 18241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:12.721097 18217 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:27:12.721195 18217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-0/wal
uuid: "638feafd83954241a681cb7e2ab8ec64"
format_stamp: "Formatted at 2026-08-08 22:27:12 on dist-test-slave-7f01"
server_key: "b1a107e378d9b003e61d76cfb3539641"
server_key_iv: "9b477372c26b763bf0b9d132541e7117"
server_key_version: "encryptionkey@0"
I20260808 22:27:12.721278 18217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:12.733830 18217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:12.737111 18217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:12.737262 18217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:12.738699 18217 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:27:12.738780 18217 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:12.738852 18217 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:27:12.738889 18217 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 22:27:12 dist-test-slave-7f01 krb5kdc[18029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228032, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM
I20260808 22:27:12.750447 18217 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.193:43025
I20260808 22:27:12.751323 18217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
I20260808 22:27:12.752142 18427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.193:43025 every 8 connection(s)
I20260808 22:27:12.753109 18175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:12.739422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:42811 (local address 127.5.11.254:43883)
0808 22:27:12.739565 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:12.739570 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:12.740637 (+ 1067us) server_negotiation.cc:407] Connection header received
0808 22:27:12.741653 (+ 1016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:12.741657 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:12.741719 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:12.741808 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:12.743434 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:12.744154 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:12.745304 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:12.745646 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:12.745768 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:12.749389 (+ 3621us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:12.749393 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:12.749398 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:12.749429 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:12.750896 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:12.751411 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:12.751415 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:12.751417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:12.751505 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:12.751948 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:12.751951 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:12.751953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:12.752185 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:12.752312 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:12.752880 (+ 568us) server_negotiation.cc:299] Negotiation successful
0808 22:27:12.752984 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 22:27:12.753849 18256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:12.739985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43883 (local address 127.5.11.193:42811)
0808 22:27:12.740477 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:12.740521 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:12.741440 (+ 919us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:12.742142 (+ 702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:12.742153 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:12.742662 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:12.743230 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:12.743243 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:12.744288 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:12.744292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:12.745168 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:12.745178 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:12.745821 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:12.745825 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:12.746000 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:12.746636 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:12.746660 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:12.749223 (+ 2563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:12.751021 (+ 1798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:12.751028 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:12.751042 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:12.751302 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:12.751602 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:12.751606 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:12.751608 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:12.751830 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:12.752329 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:12.752335 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:12.752562 (+ 227us) client_negotiation.cc:769] Sending connection context
0808 22:27:12.752774 (+ 212us) client_negotiation.cc:240] Negotiation successful
0808 22:27:12.753025 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":141,"threads_started":1}
I20260808 22:27:12.755172 18428 heartbeater.cc:347] Connected to a master server at 127.5.11.254:43883
I20260808 22:27:12.755333 18428 heartbeater.cc:464] Registering TS with master...
I20260808 22:27:12.755795 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 18217
I20260808 22:27:12.755920 18428 heartbeater.cc:511] Master 127.5.11.254:43883 requested a full tablet report, sending...
I20260808 22:27:12.755952 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance
I20260808 22:27:12.756315 5167 external_mini_cluster.cc:1428] Setting key 9b8b2dc952f39a29cc375ce59979bc6b
I20260808 22:27:12.757649 18093 ts_manager.cc:194] Registered new tserver with Master: 638feafd83954241a681cb7e2ab8ec64 (127.5.11.193:43025)
I20260808 22:27:12.759121 18093 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.11.193@KRBTEST.COM'} at 127.5.11.193:42811
I20260808 22:27:12.765100 18175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:12.757580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:58163 (local address 127.5.11.254:43883)
0808 22:27:12.757685 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:12.757690 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:12.757981 (+ 291us) server_negotiation.cc:407] Connection header received
0808 22:27:12.758103 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:12.758106 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:12.758146 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:12.758231 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:12.758793 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:12.759483 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:12.760631 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:12.760843 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:12.760953 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:12.762013 (+ 1060us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:12.762017 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:12.762019 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:12.762048 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:12.763413 (+ 1365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:12.763911 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:12.763914 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:12.763916 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:12.763996 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:12.764314 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:12.764317 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:12.764319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:12.764481 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:12.764594 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:12.764847 (+ 253us) server_negotiation.cc:299] Negotiation successful
0808 22:27:12.764952 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 22:27:12.765118 18437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:12.757486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43883 (local address 127.5.11.193:58163)
0808 22:27:12.757863 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:12.757879 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:12.757998 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:12.758278 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:12.758282 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:12.758509 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:12.758659 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:12.758670 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:12.759636 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:12.759643 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:12.760504 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:12.760514 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:12.760959 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:12.760963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:12.761073 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:12.761547 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:12.761559 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:12.761867 (+ 308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:12.763560 (+ 1693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:12.763563 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:12.763565 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:12.763807 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:12.764111 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:12.764115 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:12.764117 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:12.764215 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:12.764588 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:12.764591 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:12.764686 (+ 95us) client_negotiation.cc:769] Sending connection context
0808 22:27:12.764792 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 22:27:12.764927 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":97,"threads_started":1}
WARNING: no policy specified for oryx/127.5.11.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.11.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.11.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.194.keytab.
Entry for principal oryx/127.5.11.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.194.keytab.
I20260808 22:27:12.786728 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.194:0
--local_ip_for_outbound_sockets=127.5.11.194
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:43883
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.194.keytab
--principal=oryx/127.5.11.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:36109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:27:12.900094 18442 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:12.900334 18442 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:12.900377 18442 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:12.904314 18442 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:12.904364 18442 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:12.904402 18442 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 22:27:12.904433 18442 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.194
I20260808 22:27:12.909695 18442 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:36109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.194.keytab
--principal=oryx/127.5.11.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.11.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.11.254:43883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18442
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.194
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:12.910626 18442 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:12.911620 18442 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:12.921278 18452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:12.921257 18460 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:12.921394 18442 server_base.cc:1034] running on GCE node
W20260808 22:27:12.921262 18455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:12.921957 18442 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:12.922539 18442 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:12.923700 18442 hybrid_clock.cc:648] HybridClock initialized: now 1786228032923687 us; error 42 us; skew 500 ppm
Aug 08 22:27:12 dist-test-slave-7f01 krb5kdc[18029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228032, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:27:12.926587 18442 init.cc:377] Logged in from keytab as oryx/127.5.11.194@KRBTEST.COM (short username oryx)
I20260808 22:27:12.927430 18442 webserver.cc:460] Webserver started at http://127.5.11.194:43813/ using document root <none> and password file <none>
I20260808 22:27:12.928072 18442 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:12.928139 18442 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:12.928350 18442 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:27:12.930419 18442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-1/data/instance:
uuid: "7850c97cdd9d4c37b45acbccf1003368"
format_stamp: "Formatted at 2026-08-08 22:27:12 on dist-test-slave-7f01"
server_key: "64f3cab143d45c395da9bbac284add08"
server_key_iv: "7e978f506b08530793becc5662cad810"
server_key_version: "encryptionkey@0"
I20260808 22:27:12.931000 18442 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance:
uuid: "7850c97cdd9d4c37b45acbccf1003368"
format_stamp: "Formatted at 2026-08-08 22:27:12 on dist-test-slave-7f01"
server_key: "64f3cab143d45c395da9bbac284add08"
server_key_iv: "7e978f506b08530793becc5662cad810"
server_key_version: "encryptionkey@0"
I20260808 22:27:12.934779 18442 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260808 22:27:12.937172 18469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:12.938311 18442 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260808 22:27:12.938446 18442 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-1/wal
uuid: "7850c97cdd9d4c37b45acbccf1003368"
format_stamp: "Formatted at 2026-08-08 22:27:12 on dist-test-slave-7f01"
server_key: "64f3cab143d45c395da9bbac284add08"
server_key_iv: "7e978f506b08530793becc5662cad810"
server_key_version: "encryptionkey@0"
I20260808 22:27:12.938555 18442 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:12.959347 18442 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:12.962678 18442 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:12.962896 18442 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:12.964702 18442 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:27:12.964778 18442 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:12.964864 18442 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:27:12.964908 18442 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 22:27:12 dist-test-slave-7f01 krb5kdc[18029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228032, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM
I20260808 22:27:12.975785 18442 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.194:45435
I20260808 22:27:12.975876 18659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.194:45435 every 8 connection(s)
I20260808 22:27:12.976790 18442 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
I20260808 22:27:12.978587 18175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:12.965527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:55541 (local address 127.5.11.254:43883)
0808 22:27:12.965676 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:12.965680 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:12.966568 (+ 888us) server_negotiation.cc:407] Connection header received
0808 22:27:12.967710 (+ 1142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:12.967716 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:12.967803 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:12.967867 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:12.969644 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:12.970514 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:12.971858 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:12.972098 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:12.972219 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:12.974693 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:12.974698 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:12.974701 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:12.974731 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:12.976312 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:12.976903 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:12.976907 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:12.976909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:12.977008 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:12.977434 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:12.977440 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:12.977443 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:12.977674 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:12.977790 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:12.978279 (+ 489us) server_negotiation.cc:299] Negotiation successful
0808 22:27:12.978405 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 22:27:12.979295 18484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:12.965936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43883 (local address 127.5.11.194:55541)
0808 22:27:12.966395 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:12.966443 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:12.967436 (+ 993us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:12.968334 (+ 898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:12.968343 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:12.968745 (+ 402us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:12.969481 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:12.969505 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:12.970704 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:12.970710 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:12.971707 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:12.971717 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:12.972270 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:12.972273 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:12.972454 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:12.972969 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:12.972985 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:12.974526 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:12.976434 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:12.976443 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:12.976459 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:12.976788 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:12.977109 (+ 321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:12.977115 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:12.977119 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:12.977298 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:12.977785 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:12.977791 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:12.978046 (+ 255us) client_negotiation.cc:769] Sending connection context
0808 22:27:12.978234 (+ 188us) client_negotiation.cc:240] Negotiation successful
0808 22:27:12.978530 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":133,"threads_started":1}
I20260808 22:27:12.980643 18660 heartbeater.cc:347] Connected to a master server at 127.5.11.254:43883
I20260808 22:27:12.980810 18660 heartbeater.cc:464] Registering TS with master...
I20260808 22:27:12.981302 18660 heartbeater.cc:511] Master 127.5.11.254:43883 requested a full tablet report, sending...
I20260808 22:27:12.982332 18093 ts_manager.cc:194] Registered new tserver with Master: 7850c97cdd9d4c37b45acbccf1003368 (127.5.11.194:45435)
I20260808 22:27:12.983101 18093 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.11.194@KRBTEST.COM'} at 127.5.11.194:55541
I20260808 22:27:12.983958 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 18442
I20260808 22:27:12.984098 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance
I20260808 22:27:12.984437 5167 external_mini_cluster.cc:1428] Setting key 4ed9e09b69fe7613778391860260f722
I20260808 22:27:12.990340 18175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:12.982803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:40547 (local address 127.5.11.254:43883)
0808 22:27:12.982918 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:12.982921 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:12.983129 (+ 208us) server_negotiation.cc:407] Connection header received
0808 22:27:12.983265 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:12.983268 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:12.983330 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:12.983418 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:12.983937 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:12.984644 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:12.985588 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:12.985854 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:12.985936 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:12.987153 (+ 1217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:12.987156 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:12.987157 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:12.987179 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:12.988652 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:12.989178 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:12.989181 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:12.989182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:12.989255 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:12.989554 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:12.989556 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:12.989557 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:12.989748 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:12.989825 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:12.990152 (+ 327us) server_negotiation.cc:299] Negotiation successful
0808 22:27:12.990237 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 22:27:12.990532 18673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:12.982724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43883 (local address 127.5.11.194:40547)
0808 22:27:12.983044 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:12.983055 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:12.983173 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:12.983434 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:12.983437 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:12.983662 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:12.983797 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:12.983805 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:12.984783 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:12.984787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:12.985491 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:12.985501 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:12.985973 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:12.985977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:12.986083 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:12.986537 (+ 454us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:12.986558 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:12.987014 (+ 456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:12.988771 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:12.988776 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:12.988779 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:12.989064 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:12.989344 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:12.989349 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:12.989351 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:12.989456 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:12.989850 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:12.989856 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:12.989971 (+ 115us) client_negotiation.cc:769] Sending connection context
0808 22:27:12.990102 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 22:27:12.990268 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":119,"threads_started":1}
WARNING: no policy specified for oryx/127.5.11.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.11.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.11.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.195.keytab.
Entry for principal oryx/127.5.11.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.195.keytab.
I20260808 22:27:13.015215 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.195:0
--local_ip_for_outbound_sockets=127.5.11.195
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:43883
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.195.keytab
--principal=oryx/127.5.11.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:36109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:27:13.128583 18676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:13.128880 18676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:13.128955 18676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:13.132920 18676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:13.133013 18676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:13.133088 18676 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260808 22:27:13.133169 18676 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.195
I20260808 22:27:13.138314 18676 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:36109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.195.keytab
--principal=oryx/127.5.11.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.11.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.5.11.254:43883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.195
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:13.139366 18676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:13.140477 18676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:13.150049 18685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:13.150216 18676 server_base.cc:1034] running on GCE node
W20260808 22:27:13.150075 18689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:13.150074 18684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:13.150821 18676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:13.151468 18676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:13.152653 18676 hybrid_clock.cc:648] HybridClock initialized: now 1786228033152628 us; error 37 us; skew 500 ppm
Aug 08 22:27:13 dist-test-slave-7f01 krb5kdc[18029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228033, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:27:13.155623 18676 init.cc:377] Logged in from keytab as oryx/127.5.11.195@KRBTEST.COM (short username oryx)
I20260808 22:27:13.156545 18676 webserver.cc:460] Webserver started at http://127.5.11.195:43115/ using document root <none> and password file <none>
I20260808 22:27:13.157114 18676 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:13.157204 18676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:13.157455 18676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:27:13.159471 18676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-2/data/instance:
uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98"
format_stamp: "Formatted at 2026-08-08 22:27:13 on dist-test-slave-7f01"
server_key: "c7238be6532cd0f353132954a20f2b0b"
server_key_iv: "1baabdd417f42366e1192d9fd0dcd745"
server_key_version: "encryptionkey@0"
I20260808 22:27:13.160058 18676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance:
uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98"
format_stamp: "Formatted at 2026-08-08 22:27:13 on dist-test-slave-7f01"
server_key: "c7238be6532cd0f353132954a20f2b0b"
server_key_iv: "1baabdd417f42366e1192d9fd0dcd745"
server_key_version: "encryptionkey@0"
I20260808 22:27:13.163626 18676 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:27:13.165928 18707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:13.167110 18676 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:27:13.167244 18676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-2/wal
uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98"
format_stamp: "Formatted at 2026-08-08 22:27:13 on dist-test-slave-7f01"
server_key: "c7238be6532cd0f353132954a20f2b0b"
server_key_iv: "1baabdd417f42366e1192d9fd0dcd745"
server_key_version: "encryptionkey@0"
I20260808 22:27:13.167351 18676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:13.194921 18676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:13.198191 18676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:13.198395 18676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:13.199872 18676 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:27:13.199947 18676 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:13.200016 18676 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:27:13.200067 18676 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 08 22:27:13 dist-test-slave-7f01 krb5kdc[18029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228033, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM
I20260808 22:27:13.211891 18676 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.195:37603
I20260808 22:27:13.211956 18891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.195:37603 every 8 connection(s)
I20260808 22:27:13.212941 18676 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
I20260808 22:27:13.214421 18175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:13.200737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:41571 (local address 127.5.11.254:43883)
0808 22:27:13.200870 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:13.200874 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:13.201898 (+ 1024us) server_negotiation.cc:407] Connection header received
0808 22:27:13.203026 (+ 1128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:13.203032 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:13.203145 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:13.203237 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:13.204722 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.205317 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.206967 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.207271 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.207443 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:13.210474 (+ 3031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:13.210479 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:13.210481 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:13.210512 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:13.212281 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:13.212785 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:13.212789 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:13.212791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:13.212891 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:13.213298 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:13.213302 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:13.213303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:13.213549 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:13.213657 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:13.214196 (+ 539us) server_negotiation.cc:299] Negotiation successful
0808 22:27:13.214299 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 22:27:13.215173 18720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:13.201261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43883 (local address 127.5.11.195:41571)
0808 22:27:13.201734 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:13.201788 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:13.202759 (+ 971us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:13.203741 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:13.203777 (+ 36us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:13.204105 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:13.204565 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:13.204578 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.205452 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.205456 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:13.206815 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:13.206827 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.207483 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.207488 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:13.207690 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:13.208330 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:13.208357 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:13.210302 (+ 1945us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:13.212409 (+ 2107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:13.212414 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:13.212424 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:13.212679 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:13.213029 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:13.213035 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:13.213039 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:13.213193 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:13.213716 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:13.213724 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:13.213971 (+ 247us) client_negotiation.cc:769] Sending connection context
0808 22:27:13.214129 (+ 158us) client_negotiation.cc:240] Negotiation successful
0808 22:27:13.214356 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":124,"threads_started":1}
I20260808 22:27:13.216439 18894 heartbeater.cc:347] Connected to a master server at 127.5.11.254:43883
I20260808 22:27:13.216594 18894 heartbeater.cc:464] Registering TS with master...
I20260808 22:27:13.217106 18894 heartbeater.cc:511] Master 127.5.11.254:43883 requested a full tablet report, sending...
I20260808 22:27:13.218195 18093 ts_manager.cc:194] Registered new tserver with Master: 9fbd96f2c37b4a6a89fdca1756ab8a98 (127.5.11.195:37603)
I20260808 22:27:13.218978 18093 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.5.11.195@KRBTEST.COM'} at 127.5.11.195:41571
I20260808 22:27:13.221769 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 18676
I20260808 22:27:13.221933 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance
I20260808 22:27:13.222355 5167 external_mini_cluster.cc:1428] Setting key ed09a1cc7906fad97939037e88250121
I20260808 22:27:13.224748 5167 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:27:13.226131 18906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:13.218465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43883 (local address 127.5.11.195:40365)
0808 22:27:13.218871 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:13.218884 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:13.218980 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:13.219227 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:13.219231 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:13.219493 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:13.219614 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:13.219622 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.220645 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.220647 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:13.221190 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:13.221198 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.221643 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.221648 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:13.221751 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:13.222246 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:13.222261 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:13.222665 (+ 404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:13.224522 (+ 1857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:13.224525 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:13.224528 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:13.224817 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:13.225119 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:13.225122 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:13.225124 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:13.225212 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:13.225667 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:13.225671 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:13.225802 (+ 131us) client_negotiation.cc:769] Sending connection context
0808 22:27:13.225899 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 22:27:13.226010 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":123,"threads_started":1}
I20260808 22:27:13.226497 18175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:13.218597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:40365 (local address 127.5.11.254:43883)
0808 22:27:13.218726 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:13.218730 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:13.218960 (+ 230us) server_negotiation.cc:407] Connection header received
0808 22:27:13.219080 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:13.219082 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:13.219123 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:13.219180 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:13.219766 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.220498 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.221295 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.221524 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.221605 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:13.222819 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:13.222822 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:13.222825 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:13.222853 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:13.224409 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:13.224909 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:13.224913 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:13.224915 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:13.225009 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:13.225304 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:13.225309 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:13.225310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:13.225527 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:13.225622 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:13.226001 (+ 379us) server_negotiation.cc:299] Negotiation successful
0808 22:27:13.226136 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 22:27:13 dist-test-slave-7f01 krb5kdc[18029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228033, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 08 22:27:13 dist-test-slave-7f01 krb5kdc[18029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:13 dist-test-slave-7f01 krb5kdc[18029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
W20260808 22:27:13.250525 18175 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:13.244248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41432 (local address 127.5.11.254:43883)
0808 22:27:13.244391 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:13.244394 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:13.244406 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:27:13.244447 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:13.244449 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:13.244485 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:13.244577 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:13.245295 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.245825 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.246683 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.246916 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.247029 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:13.250394 (+ 3365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41432: BlockingRecv error: recv got EOF from 127.0.0.1:41432 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 08 22:27:13 dist-test-slave-7f01 krb5kdc[18029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228033, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM
I20260808 22:27:13.259949 18175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:13.251936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41442 (local address 127.5.11.254:43883)
0808 22:27:13.252062 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:13.252066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:13.252195 (+ 129us) server_negotiation.cc:407] Connection header received
0808 22:27:13.252288 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:13.252290 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:13.252325 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:13.252387 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:13.253050 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.253560 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.254393 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.254601 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.254708 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:13.256980 (+ 2272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:13.256983 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:13.256986 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:13.257007 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:13.258479 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:13.258850 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:13.258852 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:13.258853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:13.258927 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:13.259158 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:13.259160 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:13.259161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:13.259319 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:13.259426 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:13.259624 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 22:27:13.259711 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 22:27:13.262646 18093 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41442:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:27:13.265808 18093 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:27:13.280323 18930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:13.274233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:45435 (local address 127.0.0.1:51052)
0808 22:27:13.274978 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:13.275032 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:13.275165 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:13.275850 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:13.275857 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:13.275881 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:27:13.276022 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:13.276029 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.277423 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.277426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:13.278626 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:13.278636 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.279696 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.279700 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:13.279841 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:13.279882 (+ 41us) client_negotiation.cc:769] Sending connection context
0808 22:27:13.280039 (+ 157us) client_negotiation.cc:240] Negotiation successful
0808 22:27:13.280163 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"mutex_wait_us":64,"thread_start_us":75,"threads_started":1}
I20260808 22:27:13.280467 18938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:13.275056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47406 (local address 127.5.11.195:37603)
0808 22:27:13.275500 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:13.275514 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:13.275954 (+ 440us) server_negotiation.cc:407] Connection header received
0808 22:27:13.276038 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:13.276043 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:13.276227 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:13.276378 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:27:13.276587 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.277357 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.279033 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.279727 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.279884 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:13.280042 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:13.280230 (+ 188us) server_negotiation.cc:299] Negotiation successful
0808 22:27:13.280319 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":111,"threads_started":1}
I20260808 22:27:13.280789 18934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:13.274430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51052 (local address 127.5.11.194:45435)
0808 22:27:13.274789 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:13.274804 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:13.275312 (+ 508us) server_negotiation.cc:407] Connection header received
0808 22:27:13.275409 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:13.275414 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:13.275652 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:13.275862 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:27:13.276150 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.277289 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.278751 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.279317 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.279447 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:13.279623 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:13.280543 (+ 920us) server_negotiation.cc:299] Negotiation successful
0808 22:27:13.280646 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":143,"threads_started":1}
I20260808 22:27:13.280790 18936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:13.275244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:43025 (local address 127.0.0.1:35954)
0808 22:27:13.275726 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:13.275764 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:13.275843 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:13.276574 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:13.276577 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:13.276596 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:27:13.276680 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:13.276686 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.278170 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.278173 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:13.279299 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:13.279309 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.280428 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.280434 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:13.280548 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:13.280574 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 22:27:13.280625 (+ 51us) client_negotiation.cc:240] Negotiation successful
0808 22:27:13.280683 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"mutex_wait_us":1,"thread_start_us":103,"threads_started":1}
I20260808 22:27:13.280902 18939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:13.274917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:37603 (local address 127.0.0.1:47406)
0808 22:27:13.275490 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:13.275505 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:13.275610 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:13.276356 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:13.276359 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:13.276376 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:27:13.276482 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:13.276492 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.277508 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.277511 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:13.278899 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:13.278910 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.279908 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.279912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:13.280073 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:13.280096 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 22:27:13.280197 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 22:27:13.280267 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"mutex_wait_us":72,"thread_start_us":96,"threads_started":1}
I20260808 22:27:13.281366 18940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:13.275668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35954 (local address 127.5.11.193:43025)
0808 22:27:13.276119 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:13.276128 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:13.276150 (+ 22us) server_negotiation.cc:407] Connection header received
0808 22:27:13.276233 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:13.276238 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:13.276419 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:13.276829 (+ 410us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:27:13.276960 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.278038 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.279432 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.280279 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.280899 (+ 620us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:13.281054 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:13.281112 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 22:27:13.281205 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":112,"threads_started":1}
I20260808 22:27:13.283865 18546 tablet_service.cc:1467] Processing CreateTablet for tablet 751b42c7dabe48089c5b503f4b5ea0c6 (DEFAULT_TABLE table=test-table [id=986212911df14d0295033e42e993d7da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:27:13.284005 18775 tablet_service.cc:1467] Processing CreateTablet for tablet 751b42c7dabe48089c5b503f4b5ea0c6 (DEFAULT_TABLE table=test-table [id=986212911df14d0295033e42e993d7da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:27:13.283993 18327 tablet_service.cc:1467] Processing CreateTablet for tablet 751b42c7dabe48089c5b503f4b5ea0c6 (DEFAULT_TABLE table=test-table [id=986212911df14d0295033e42e993d7da]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:27:13.284930 18546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 751b42c7dabe48089c5b503f4b5ea0c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:27:13.285044 18775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 751b42c7dabe48089c5b503f4b5ea0c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:27:13.285044 18327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 751b42c7dabe48089c5b503f4b5ea0c6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:27:13.291950 18941 tablet_bootstrap.cc:492] T 751b42c7dabe48089c5b503f4b5ea0c6 P 7850c97cdd9d4c37b45acbccf1003368: Bootstrap starting.
I20260808 22:27:13.293166 18942 tablet_bootstrap.cc:492] T 751b42c7dabe48089c5b503f4b5ea0c6 P 638feafd83954241a681cb7e2ab8ec64: Bootstrap starting.
I20260808 22:27:13.294168 18941 tablet_bootstrap.cc:654] T 751b42c7dabe48089c5b503f4b5ea0c6 P 7850c97cdd9d4c37b45acbccf1003368: Neither blocks nor log segments found. Creating new log.
I20260808 22:27:13.294898 18942 tablet_bootstrap.cc:654] T 751b42c7dabe48089c5b503f4b5ea0c6 P 638feafd83954241a681cb7e2ab8ec64: Neither blocks nor log segments found. Creating new log.
I20260808 22:27:13.294950 18941 log.cc:824] T 751b42c7dabe48089c5b503f4b5ea0c6 P 7850c97cdd9d4c37b45acbccf1003368: Log is configured to *not* fsync() on all Append() calls
I20260808 22:27:13.295630 18943 tablet_bootstrap.cc:492] T 751b42c7dabe48089c5b503f4b5ea0c6 P 9fbd96f2c37b4a6a89fdca1756ab8a98: Bootstrap starting.
I20260808 22:27:13.295753 18942 log.cc:824] T 751b42c7dabe48089c5b503f4b5ea0c6 P 638feafd83954241a681cb7e2ab8ec64: Log is configured to *not* fsync() on all Append() calls
I20260808 22:27:13.297094 18941 tablet_bootstrap.cc:492] T 751b42c7dabe48089c5b503f4b5ea0c6 P 7850c97cdd9d4c37b45acbccf1003368: No bootstrap required, opened a new log
I20260808 22:27:13.297310 18941 ts_tablet_manager.cc:1397] T 751b42c7dabe48089c5b503f4b5ea0c6 P 7850c97cdd9d4c37b45acbccf1003368: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 22:27:13.297776 18942 tablet_bootstrap.cc:492] T 751b42c7dabe48089c5b503f4b5ea0c6 P 638feafd83954241a681cb7e2ab8ec64: No bootstrap required, opened a new log
I20260808 22:27:13.297986 18942 ts_tablet_manager.cc:1397] T 751b42c7dabe48089c5b503f4b5ea0c6 P 638feafd83954241a681cb7e2ab8ec64: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 22:27:13.297966 18943 tablet_bootstrap.cc:654] T 751b42c7dabe48089c5b503f4b5ea0c6 P 9fbd96f2c37b4a6a89fdca1756ab8a98: Neither blocks nor log segments found. Creating new log.
I20260808 22:27:13.298779 18943 log.cc:824] T 751b42c7dabe48089c5b503f4b5ea0c6 P 9fbd96f2c37b4a6a89fdca1756ab8a98: Log is configured to *not* fsync() on all Append() calls
I20260808 22:27:13.300609 18941 raft_consensus.cc:348] T 751b42c7dabe48089c5b503f4b5ea0c6 P 7850c97cdd9d4c37b45acbccf1003368 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } } peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } }
I20260808 22:27:13.300845 18943 tablet_bootstrap.cc:492] T 751b42c7dabe48089c5b503f4b5ea0c6 P 9fbd96f2c37b4a6a89fdca1756ab8a98: No bootstrap required, opened a new log
I20260808 22:27:13.300912 18941 raft_consensus.cc:374] T 751b42c7dabe48089c5b503f4b5ea0c6 P 7850c97cdd9d4c37b45acbccf1003368 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:27:13.301014 18941 raft_consensus.cc:729] T 751b42c7dabe48089c5b503f4b5ea0c6 P 7850c97cdd9d4c37b45acbccf1003368 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7850c97cdd9d4c37b45acbccf1003368, State: Initialized, Role: FOLLOWER
I20260808 22:27:13.301031 18943 ts_tablet_manager.cc:1397] T 751b42c7dabe48089c5b503f4b5ea0c6 P 9fbd96f2c37b4a6a89fdca1756ab8a98: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 22:27:13.301497 18941 consensus_queue.cc:260] T 751b42c7dabe48089c5b503f4b5ea0c6 P 7850c97cdd9d4c37b45acbccf1003368 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } } peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } }
I20260808 22:27:13.302148 18660 heartbeater.cc:503] Master 127.5.11.254:43883 was elected leader, sending a full tablet report...
I20260808 22:27:13.302516 18941 ts_tablet_manager.cc:1428] T 751b42c7dabe48089c5b503f4b5ea0c6 P 7850c97cdd9d4c37b45acbccf1003368: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 22:27:13.302469 18942 raft_consensus.cc:348] T 751b42c7dabe48089c5b503f4b5ea0c6 P 638feafd83954241a681cb7e2ab8ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } } peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } }
I20260808 22:27:13.302732 18942 raft_consensus.cc:374] T 751b42c7dabe48089c5b503f4b5ea0c6 P 638feafd83954241a681cb7e2ab8ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:27:13.302786 18942 raft_consensus.cc:729] T 751b42c7dabe48089c5b503f4b5ea0c6 P 638feafd83954241a681cb7e2ab8ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 638feafd83954241a681cb7e2ab8ec64, State: Initialized, Role: FOLLOWER
I20260808 22:27:13.303234 18942 consensus_queue.cc:260] T 751b42c7dabe48089c5b503f4b5ea0c6 P 638feafd83954241a681cb7e2ab8ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } } peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } }
I20260808 22:27:13.304275 18942 ts_tablet_manager.cc:1428] T 751b42c7dabe48089c5b503f4b5ea0c6 P 638feafd83954241a681cb7e2ab8ec64: Time spent starting tablet: real 0.006s user 0.002s sys 0.005s
I20260808 22:27:13.304813 18428 heartbeater.cc:503] Master 127.5.11.254:43883 was elected leader, sending a full tablet report...
I20260808 22:27:13.305114 18943 raft_consensus.cc:348] T 751b42c7dabe48089c5b503f4b5ea0c6 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } } peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } }
I20260808 22:27:13.305385 18943 raft_consensus.cc:374] T 751b42c7dabe48089c5b503f4b5ea0c6 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:27:13.305452 18943 raft_consensus.cc:729] T 751b42c7dabe48089c5b503f4b5ea0c6 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fbd96f2c37b4a6a89fdca1756ab8a98, State: Initialized, Role: FOLLOWER
I20260808 22:27:13.305851 18943 consensus_queue.cc:260] T 751b42c7dabe48089c5b503f4b5ea0c6 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } } peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } }
I20260808 22:27:13.306679 18894 heartbeater.cc:503] Master 127.5.11.254:43883 was elected leader, sending a full tablet report...
I20260808 22:27:13.307021 18943 ts_tablet_manager.cc:1428] T 751b42c7dabe48089c5b503f4b5ea0c6 P 9fbd96f2c37b4a6a89fdca1756ab8a98: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260808 22:27:13.464092 18896 tablet.cc:2369] T 751b42c7dabe48089c5b503f4b5ea0c6 P 9fbd96f2c37b4a6a89fdca1756ab8a98: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 22:27:13.478035 18664 tablet.cc:2369] T 751b42c7dabe48089c5b503f4b5ea0c6 P 7850c97cdd9d4c37b45acbccf1003368: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 22:27:13.502841 18429 tablet.cc:2369] T 751b42c7dabe48089c5b503f4b5ea0c6 P 638feafd83954241a681cb7e2ab8ec64: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:27:13.556563 18950 raft_consensus.cc:482] T 751b42c7dabe48089c5b503f4b5ea0c6 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:27:13.556782 18950 raft_consensus.cc:504] T 751b42c7dabe48089c5b503f4b5ea0c6 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } } peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } }
I20260808 22:27:13.557969 18950 leader_election.cc:288] T 751b42c7dabe48089c5b503f4b5ea0c6 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7850c97cdd9d4c37b45acbccf1003368 (127.5.11.194:45435), 638feafd83954241a681cb7e2ab8ec64 (127.5.11.193:43025)
I20260808 22:27:13.562630 18720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:13.558307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:45435 (local address 127.5.11.195:54379)
0808 22:27:13.558462 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:13.558485 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:13.558591 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:13.558905 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:13.558911 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:13.558935 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:27:13.559083 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:13.559089 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.560197 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.560200 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:13.561366 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:13.561376 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.562199 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.562202 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:13.562309 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:13.562336 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 22:27:13.562416 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 22:27:13.562489 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260808 22:27:13.562659 18934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:13.558378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:54379 (local address 127.5.11.194:45435)
0808 22:27:13.558572 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:13.558579 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:13.558602 (+ 23us) server_negotiation.cc:407] Connection header received
0808 22:27:13.558708 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:13.558711 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:13.558775 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:13.558872 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:27:13.559217 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.560079 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.561502 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.562086 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.562203 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:13.562257 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:13.562447 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 22:27:13.562528 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 22:27:13.563356 18576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "751b42c7dabe48089c5b503f4b5ea0c6" candidate_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7850c97cdd9d4c37b45acbccf1003368" is_pre_election: true
I20260808 22:27:13.563716 18951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:13.558689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:43025 (local address 127.5.11.195:56727)
0808 22:27:13.559132 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:13.559166 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:13.559276 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:13.559518 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:13.559521 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:13.559541 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:27:13.560136 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:13.560149 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.561156 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.561159 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:13.562233 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:13.562243 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.563256 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.563260 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:13.563359 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:13.563382 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 22:27:13.563498 (+ 116us) client_negotiation.cc:240] Negotiation successful
0808 22:27:13.563569 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"mutex_wait_us":1,"thread_start_us":128,"threads_started":1}
I20260808 22:27:13.563706 18940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:13.558882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:56727 (local address 127.5.11.193:43025)
0808 22:27:13.559013 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:13.559017 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:13.559265 (+ 248us) server_negotiation.cc:407] Connection header received
0808 22:27:13.559364 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:13.559366 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:13.559412 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:13.559473 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:27:13.560278 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.561034 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.562417 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.563013 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.563131 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:13.563179 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:13.563516 (+ 337us) server_negotiation.cc:299] Negotiation successful
0808 22:27:13.563597 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 22:27:13.563730 18576 raft_consensus.cc:2454] T 751b42c7dabe48089c5b503f4b5ea0c6 P 7850c97cdd9d4c37b45acbccf1003368 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9fbd96f2c37b4a6a89fdca1756ab8a98 in term 0.
I20260808 22:27:13.564245 18711 leader_election.cc:302] T 751b42c7dabe48089c5b503f4b5ea0c6 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7850c97cdd9d4c37b45acbccf1003368, 9fbd96f2c37b4a6a89fdca1756ab8a98; no voters:
I20260808 22:27:13.564548 18950 raft_consensus.cc:2781] T 751b42c7dabe48089c5b503f4b5ea0c6 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:27:13.564510 18355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "751b42c7dabe48089c5b503f4b5ea0c6" candidate_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "638feafd83954241a681cb7e2ab8ec64" is_pre_election: true
I20260808 22:27:13.564653 18950 raft_consensus.cc:482] T 751b42c7dabe48089c5b503f4b5ea0c6 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:27:13.564704 18950 raft_consensus.cc:3037] T 751b42c7dabe48089c5b503f4b5ea0c6 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:27:13.564821 18355 raft_consensus.cc:2454] T 751b42c7dabe48089c5b503f4b5ea0c6 P 638feafd83954241a681cb7e2ab8ec64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9fbd96f2c37b4a6a89fdca1756ab8a98 in term 0.
I20260808 22:27:13.565868 18950 raft_consensus.cc:504] T 751b42c7dabe48089c5b503f4b5ea0c6 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } } peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } }
I20260808 22:27:13.566161 18950 leader_election.cc:288] T 751b42c7dabe48089c5b503f4b5ea0c6 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [CANDIDATE]: Term 1 election: Requested vote from peers 7850c97cdd9d4c37b45acbccf1003368 (127.5.11.194:45435), 638feafd83954241a681cb7e2ab8ec64 (127.5.11.193:43025)
I20260808 22:27:13.566546 18576 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "751b42c7dabe48089c5b503f4b5ea0c6" candidate_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7850c97cdd9d4c37b45acbccf1003368"
I20260808 22:27:13.566624 18355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "751b42c7dabe48089c5b503f4b5ea0c6" candidate_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "638feafd83954241a681cb7e2ab8ec64"
I20260808 22:27:13.566689 18576 raft_consensus.cc:3037] T 751b42c7dabe48089c5b503f4b5ea0c6 P 7850c97cdd9d4c37b45acbccf1003368 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:27:13.566785 18355 raft_consensus.cc:3037] T 751b42c7dabe48089c5b503f4b5ea0c6 P 638feafd83954241a681cb7e2ab8ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:27:13.567765 18576 raft_consensus.cc:2454] T 751b42c7dabe48089c5b503f4b5ea0c6 P 7850c97cdd9d4c37b45acbccf1003368 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fbd96f2c37b4a6a89fdca1756ab8a98 in term 1.
I20260808 22:27:13.567957 18355 raft_consensus.cc:2454] T 751b42c7dabe48089c5b503f4b5ea0c6 P 638feafd83954241a681cb7e2ab8ec64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fbd96f2c37b4a6a89fdca1756ab8a98 in term 1.
I20260808 22:27:13.568068 18711 leader_election.cc:302] T 751b42c7dabe48089c5b503f4b5ea0c6 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7850c97cdd9d4c37b45acbccf1003368, 9fbd96f2c37b4a6a89fdca1756ab8a98; no voters:
I20260808 22:27:13.568229 18950 raft_consensus.cc:2781] T 751b42c7dabe48089c5b503f4b5ea0c6 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:27:13.568429 18950 raft_consensus.cc:686] T 751b42c7dabe48089c5b503f4b5ea0c6 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 1 LEADER]: Becoming Leader. State: Replica: 9fbd96f2c37b4a6a89fdca1756ab8a98, State: Running, Role: LEADER
I20260808 22:27:13.568770 18950 consensus_queue.cc:237] T 751b42c7dabe48089c5b503f4b5ea0c6 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } } peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } }
I20260808 22:27:13.572077 18088 catalog_manager.cc:5495] T 751b42c7dabe48089c5b503f4b5ea0c6 P 9fbd96f2c37b4a6a89fdca1756ab8a98 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9fbd96f2c37b4a6a89fdca1756ab8a98 (127.5.11.195). New cstate: current_term: 1 leader_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } health_report { overall_health: UNKNOWN } } }
I20260808 22:27:13.642710 18938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:13.639275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47408 (local address 127.5.11.195:37603)
0808 22:27:13.639409 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:13.639413 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:13.639426 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:27:13.639513 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:13.639516 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:13.639574 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:13.639668 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:27:13.639951 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.640459 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.641258 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.641469 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.641551 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:13.641779 (+ 228us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:27:13.642218 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:27:13.642386 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:13.642466 (+ 80us) server_negotiation.cc:299] Negotiation successful
0808 22:27:13.642523 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 22:27:13.648671 18355 raft_consensus.cc:1261] T 751b42c7dabe48089c5b503f4b5ea0c6 P 638feafd83954241a681cb7e2ab8ec64 [term 1 FOLLOWER]: Refusing update from remote peer 9fbd96f2c37b4a6a89fdca1756ab8a98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:27:13.648692 18576 raft_consensus.cc:1261] T 751b42c7dabe48089c5b503f4b5ea0c6 P 7850c97cdd9d4c37b45acbccf1003368 [term 1 FOLLOWER]: Refusing update from remote peer 9fbd96f2c37b4a6a89fdca1756ab8a98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:27:13.649415 18952 consensus_queue.cc:1035] T 751b42c7dabe48089c5b503f4b5ea0c6 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:27:13.649634 18950 consensus_queue.cc:1035] T 751b42c7dabe48089c5b503f4b5ea0c6 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:27:13.658663 18955 mvcc.cc:205] Tried to move back new op lower bound from 7316390025817268224 to 7316390025500270592. Current Snapshot: MvccSnapshot[applied={T|T < 7316390025817268224 or (T in {7316390025817268224})}]
I20260808 22:27:13.660691 18956 mvcc.cc:205] Tried to move back new op lower bound from 7316390025817268224 to 7316390025500270592. Current Snapshot: MvccSnapshot[applied={T|T < 7316390025817268224}]
I20260808 22:27:13.665933 18088 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:27:13.666113 18088 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:27:13.668437 18088 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 258796a613b547bb87a6fa5fe06e6556: Sending DeleteTablet for 3 replicas of tablet 751b42c7dabe48089c5b503f4b5ea0c6
I20260808 22:27:13.669152 18546 tablet_service.cc:1514] Processing DeleteTablet for tablet 751b42c7dabe48089c5b503f4b5ea0c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:27:13 UTC) from {username='oryx', principal='oryx/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:51052
I20260808 22:27:13.669462 18775 tablet_service.cc:1514] Processing DeleteTablet for tablet 751b42c7dabe48089c5b503f4b5ea0c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:27:13 UTC) from {username='oryx', principal='oryx/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:47406
I20260808 22:27:13.669899 18327 tablet_service.cc:1514] Processing DeleteTablet for tablet 751b42c7dabe48089c5b503f4b5ea0c6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:27:13 UTC) from {username='oryx', principal='oryx/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:35954
I20260808 22:27:13.669979 18088 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41442:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 22:27:13.670043 18967 tablet_replica.cc:330] T 751b42c7dabe48089c5b503f4b5ea0c6 P 7850c97cdd9d4c37b45acbccf1003368: stopping tablet replica
W20260808 22:27:13.670389 18088 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:27:13.670421 18967 raft_consensus.cc:2229] T 751b42c7dabe48089c5b503f4b5ea0c6 P 7850c97cdd9d4c37b45acbccf1003368 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:27:13.670701 18968 tablet_replica.cc:330] T 751b42c7dabe48089c5b503f4b5ea0c6 P 9fbd96f2c37b4a6a89fdca1756ab8a98: stopping tablet replica
I20260808 22:27:13.670862 18967 raft_consensus.cc:2258] T 751b42c7dabe48089c5b503f4b5ea0c6 P 7850c97cdd9d4c37b45acbccf1003368 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:27:13.671061 18968 raft_consensus.cc:2229] T 751b42c7dabe48089c5b503f4b5ea0c6 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:27:13.671604 18968 raft_consensus.cc:2258] T 751b42c7dabe48089c5b503f4b5ea0c6 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:27:13.672487 18967 ts_tablet_manager.cc:1905] T 751b42c7dabe48089c5b503f4b5ea0c6 P 7850c97cdd9d4c37b45acbccf1003368: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:27:13.673087 18968 ts_tablet_manager.cc:1905] T 751b42c7dabe48089c5b503f4b5ea0c6 P 9fbd96f2c37b4a6a89fdca1756ab8a98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:27:13.675347 18967 ts_tablet_manager.cc:1918] T 751b42c7dabe48089c5b503f4b5ea0c6 P 7850c97cdd9d4c37b45acbccf1003368: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:27:13.675518 18967 log.cc:1196] T 751b42c7dabe48089c5b503f4b5ea0c6 P 7850c97cdd9d4c37b45acbccf1003368: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-1/wal/wals/751b42c7dabe48089c5b503f4b5ea0c6
I20260808 22:27:13.675935 18967 ts_tablet_manager.cc:1939] T 751b42c7dabe48089c5b503f4b5ea0c6 P 7850c97cdd9d4c37b45acbccf1003368: Deleting consensus metadata
I20260808 22:27:13.676123 18775 tablet_service.cc:1467] Processing CreateTablet for tablet b98873446d8a4f47a921298d4ba14fea (DEFAULT_TABLE table=test-table [id=440d887cb8604c48967b7ea8cce87206]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:27:13.676124 18327 tablet_service.cc:1467] Processing CreateTablet for tablet b98873446d8a4f47a921298d4ba14fea (DEFAULT_TABLE table=test-table [id=440d887cb8604c48967b7ea8cce87206]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:27:13.676370 18775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b98873446d8a4f47a921298d4ba14fea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:27:13.676370 18327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b98873446d8a4f47a921298d4ba14fea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:27:13.678606 18943 tablet_bootstrap.cc:492] T b98873446d8a4f47a921298d4ba14fea P 9fbd96f2c37b4a6a89fdca1756ab8a98: Bootstrap starting.
I20260808 22:27:13.678676 18968 ts_tablet_manager.cc:1918] T 751b42c7dabe48089c5b503f4b5ea0c6 P 9fbd96f2c37b4a6a89fdca1756ab8a98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:27:13.678777 18968 log.cc:1196] T 751b42c7dabe48089c5b503f4b5ea0c6 P 9fbd96f2c37b4a6a89fdca1756ab8a98: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-2/wal/wals/751b42c7dabe48089c5b503f4b5ea0c6
I20260808 22:27:13.678910 18546 tablet_service.cc:1467] Processing CreateTablet for tablet b98873446d8a4f47a921298d4ba14fea (DEFAULT_TABLE table=test-table [id=440d887cb8604c48967b7ea8cce87206]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:27:13.679123 18968 ts_tablet_manager.cc:1939] T 751b42c7dabe48089c5b503f4b5ea0c6 P 9fbd96f2c37b4a6a89fdca1756ab8a98: Deleting consensus metadata
I20260808 22:27:13.679167 18546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b98873446d8a4f47a921298d4ba14fea. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:27:13.679664 18943 tablet_bootstrap.cc:654] T b98873446d8a4f47a921298d4ba14fea P 9fbd96f2c37b4a6a89fdca1756ab8a98: Neither blocks nor log segments found. Creating new log.
I20260808 22:27:13.680341 18942 tablet_bootstrap.cc:492] T b98873446d8a4f47a921298d4ba14fea P 638feafd83954241a681cb7e2ab8ec64: Bootstrap starting.
I20260808 22:27:13.681296 18942 tablet_bootstrap.cc:654] T b98873446d8a4f47a921298d4ba14fea P 638feafd83954241a681cb7e2ab8ec64: Neither blocks nor log segments found. Creating new log.
I20260808 22:27:13.681344 18941 tablet_bootstrap.cc:492] T b98873446d8a4f47a921298d4ba14fea P 7850c97cdd9d4c37b45acbccf1003368: Bootstrap starting.
I20260808 22:27:13.681872 18073 catalog_manager.cc:4841] TS 7850c97cdd9d4c37b45acbccf1003368 (127.5.11.194:45435): tablet 751b42c7dabe48089c5b503f4b5ea0c6 (table test-table [id=986212911df14d0295033e42e993d7da]) successfully deleted
I20260808 22:27:13.682130 18073 catalog_manager.cc:4841] TS 9fbd96f2c37b4a6a89fdca1756ab8a98 (127.5.11.195:37603): tablet 751b42c7dabe48089c5b503f4b5ea0c6 (table test-table [id=986212911df14d0295033e42e993d7da]) successfully deleted
I20260808 22:27:13.682305 18941 tablet_bootstrap.cc:654] T b98873446d8a4f47a921298d4ba14fea P 7850c97cdd9d4c37b45acbccf1003368: Neither blocks nor log segments found. Creating new log.
I20260808 22:27:13.682449 18943 tablet_bootstrap.cc:492] T b98873446d8a4f47a921298d4ba14fea P 9fbd96f2c37b4a6a89fdca1756ab8a98: No bootstrap required, opened a new log
I20260808 22:27:13.682521 18943 ts_tablet_manager.cc:1397] T b98873446d8a4f47a921298d4ba14fea P 9fbd96f2c37b4a6a89fdca1756ab8a98: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.002s
I20260808 22:27:13.682940 18943 raft_consensus.cc:348] T b98873446d8a4f47a921298d4ba14fea P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } }
I20260808 22:27:13.683041 18943 raft_consensus.cc:374] T b98873446d8a4f47a921298d4ba14fea P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:27:13.683072 18943 raft_consensus.cc:729] T b98873446d8a4f47a921298d4ba14fea P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fbd96f2c37b4a6a89fdca1756ab8a98, State: Initialized, Role: FOLLOWER
I20260808 22:27:13.683187 18943 consensus_queue.cc:260] T b98873446d8a4f47a921298d4ba14fea P 9fbd96f2c37b4a6a89fdca1756ab8a98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } }
I20260808 22:27:13.683821 18969 tablet_replica.cc:330] T 751b42c7dabe48089c5b503f4b5ea0c6 P 638feafd83954241a681cb7e2ab8ec64: stopping tablet replica
I20260808 22:27:13.684016 18941 tablet_bootstrap.cc:492] T b98873446d8a4f47a921298d4ba14fea P 7850c97cdd9d4c37b45acbccf1003368: No bootstrap required, opened a new log
I20260808 22:27:13.684096 18941 ts_tablet_manager.cc:1397] T b98873446d8a4f47a921298d4ba14fea P 7850c97cdd9d4c37b45acbccf1003368: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20260808 22:27:13.684185 18969 raft_consensus.cc:2229] T 751b42c7dabe48089c5b503f4b5ea0c6 P 638feafd83954241a681cb7e2ab8ec64 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:27:13.684507 18969 raft_consensus.cc:2258] T 751b42c7dabe48089c5b503f4b5ea0c6 P 638feafd83954241a681cb7e2ab8ec64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:27:13.684511 18941 raft_consensus.cc:348] T b98873446d8a4f47a921298d4ba14fea P 7850c97cdd9d4c37b45acbccf1003368 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } }
I20260808 22:27:13.684608 18941 raft_consensus.cc:374] T b98873446d8a4f47a921298d4ba14fea P 7850c97cdd9d4c37b45acbccf1003368 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:27:13.684641 18941 raft_consensus.cc:729] T b98873446d8a4f47a921298d4ba14fea P 7850c97cdd9d4c37b45acbccf1003368 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7850c97cdd9d4c37b45acbccf1003368, State: Initialized, Role: FOLLOWER
I20260808 22:27:13.684759 18941 consensus_queue.cc:260] T b98873446d8a4f47a921298d4ba14fea P 7850c97cdd9d4c37b45acbccf1003368 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } }
I20260808 22:27:13.685076 18941 ts_tablet_manager.cc:1428] T b98873446d8a4f47a921298d4ba14fea P 7850c97cdd9d4c37b45acbccf1003368: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 22:27:13.685105 18943 ts_tablet_manager.cc:1428] T b98873446d8a4f47a921298d4ba14fea P 9fbd96f2c37b4a6a89fdca1756ab8a98: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20260808 22:27:13.685865 18942 tablet_bootstrap.cc:492] T b98873446d8a4f47a921298d4ba14fea P 638feafd83954241a681cb7e2ab8ec64: No bootstrap required, opened a new log
I20260808 22:27:13.685953 18942 ts_tablet_manager.cc:1397] T b98873446d8a4f47a921298d4ba14fea P 638feafd83954241a681cb7e2ab8ec64: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260808 22:27:13.686375 18942 raft_consensus.cc:348] T b98873446d8a4f47a921298d4ba14fea P 638feafd83954241a681cb7e2ab8ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } }
I20260808 22:27:13.686477 18942 raft_consensus.cc:374] T b98873446d8a4f47a921298d4ba14fea P 638feafd83954241a681cb7e2ab8ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:27:13.686506 18942 raft_consensus.cc:729] T b98873446d8a4f47a921298d4ba14fea P 638feafd83954241a681cb7e2ab8ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 638feafd83954241a681cb7e2ab8ec64, State: Initialized, Role: FOLLOWER
I20260808 22:27:13.686612 18942 consensus_queue.cc:260] T b98873446d8a4f47a921298d4ba14fea P 638feafd83954241a681cb7e2ab8ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } }
I20260808 22:27:13.686853 18942 ts_tablet_manager.cc:1428] T b98873446d8a4f47a921298d4ba14fea P 638feafd83954241a681cb7e2ab8ec64: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:27:13.689018 18969 ts_tablet_manager.cc:1905] T 751b42c7dabe48089c5b503f4b5ea0c6 P 638feafd83954241a681cb7e2ab8ec64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:27:13.691687 18969 ts_tablet_manager.cc:1918] T 751b42c7dabe48089c5b503f4b5ea0c6 P 638feafd83954241a681cb7e2ab8ec64: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:27:13.691849 18969 log.cc:1196] T 751b42c7dabe48089c5b503f4b5ea0c6 P 638feafd83954241a681cb7e2ab8ec64: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-0/wal/wals/751b42c7dabe48089c5b503f4b5ea0c6
I20260808 22:27:13.692152 18969 ts_tablet_manager.cc:1939] T 751b42c7dabe48089c5b503f4b5ea0c6 P 638feafd83954241a681cb7e2ab8ec64: Deleting consensus metadata
I20260808 22:27:13.693185 18073 catalog_manager.cc:4841] TS 638feafd83954241a681cb7e2ab8ec64 (127.5.11.193:43025): tablet 751b42c7dabe48089c5b503f4b5ea0c6 (table test-table [id=986212911df14d0295033e42e993d7da]) successfully deleted
I20260808 22:27:13.771790 18950 raft_consensus.cc:482] T b98873446d8a4f47a921298d4ba14fea P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:27:13.771963 18950 raft_consensus.cc:504] T b98873446d8a4f47a921298d4ba14fea P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } }
I20260808 22:27:13.772395 18950 leader_election.cc:288] T b98873446d8a4f47a921298d4ba14fea P 9fbd96f2c37b4a6a89fdca1756ab8a98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 638feafd83954241a681cb7e2ab8ec64 (127.5.11.193:43025), 7850c97cdd9d4c37b45acbccf1003368 (127.5.11.194:45435)
I20260808 22:27:13.772856 18355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b98873446d8a4f47a921298d4ba14fea" candidate_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "638feafd83954241a681cb7e2ab8ec64" is_pre_election: true
I20260808 22:27:13.772892 18572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b98873446d8a4f47a921298d4ba14fea" candidate_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7850c97cdd9d4c37b45acbccf1003368" is_pre_election: true
I20260808 22:27:13.773020 18355 raft_consensus.cc:2454] T b98873446d8a4f47a921298d4ba14fea P 638feafd83954241a681cb7e2ab8ec64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9fbd96f2c37b4a6a89fdca1756ab8a98 in term 0.
I20260808 22:27:13.773084 18572 raft_consensus.cc:2454] T b98873446d8a4f47a921298d4ba14fea P 7850c97cdd9d4c37b45acbccf1003368 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9fbd96f2c37b4a6a89fdca1756ab8a98 in term 0.
I20260808 22:27:13.773317 18711 leader_election.cc:302] T b98873446d8a4f47a921298d4ba14fea P 9fbd96f2c37b4a6a89fdca1756ab8a98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 638feafd83954241a681cb7e2ab8ec64, 9fbd96f2c37b4a6a89fdca1756ab8a98; no voters:
I20260808 22:27:13.773520 18950 raft_consensus.cc:2781] T b98873446d8a4f47a921298d4ba14fea P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:27:13.773597 18950 raft_consensus.cc:482] T b98873446d8a4f47a921298d4ba14fea P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:27:13.773650 18950 raft_consensus.cc:3037] T b98873446d8a4f47a921298d4ba14fea P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:27:13.774657 18950 raft_consensus.cc:504] T b98873446d8a4f47a921298d4ba14fea P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } }
I20260808 22:27:13.774977 18950 leader_election.cc:288] T b98873446d8a4f47a921298d4ba14fea P 9fbd96f2c37b4a6a89fdca1756ab8a98 [CANDIDATE]: Term 1 election: Requested vote from peers 638feafd83954241a681cb7e2ab8ec64 (127.5.11.193:43025), 7850c97cdd9d4c37b45acbccf1003368 (127.5.11.194:45435)
I20260808 22:27:13.775345 18355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b98873446d8a4f47a921298d4ba14fea" candidate_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "638feafd83954241a681cb7e2ab8ec64"
I20260808 22:27:13.775362 18572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b98873446d8a4f47a921298d4ba14fea" candidate_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7850c97cdd9d4c37b45acbccf1003368"
I20260808 22:27:13.775472 18355 raft_consensus.cc:3037] T b98873446d8a4f47a921298d4ba14fea P 638feafd83954241a681cb7e2ab8ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:27:13.775498 18572 raft_consensus.cc:3037] T b98873446d8a4f47a921298d4ba14fea P 7850c97cdd9d4c37b45acbccf1003368 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:27:13.776541 18572 raft_consensus.cc:2454] T b98873446d8a4f47a921298d4ba14fea P 7850c97cdd9d4c37b45acbccf1003368 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fbd96f2c37b4a6a89fdca1756ab8a98 in term 1.
I20260808 22:27:13.776656 18355 raft_consensus.cc:2454] T b98873446d8a4f47a921298d4ba14fea P 638feafd83954241a681cb7e2ab8ec64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fbd96f2c37b4a6a89fdca1756ab8a98 in term 1.
I20260808 22:27:13.776821 18711 leader_election.cc:302] T b98873446d8a4f47a921298d4ba14fea P 9fbd96f2c37b4a6a89fdca1756ab8a98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7850c97cdd9d4c37b45acbccf1003368, 9fbd96f2c37b4a6a89fdca1756ab8a98; no voters:
I20260808 22:27:13.776991 18950 raft_consensus.cc:2781] T b98873446d8a4f47a921298d4ba14fea P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:27:13.777127 18950 raft_consensus.cc:686] T b98873446d8a4f47a921298d4ba14fea P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 1 LEADER]: Becoming Leader. State: Replica: 9fbd96f2c37b4a6a89fdca1756ab8a98, State: Running, Role: LEADER
I20260808 22:27:13.777281 18950 consensus_queue.cc:237] T b98873446d8a4f47a921298d4ba14fea P 9fbd96f2c37b4a6a89fdca1756ab8a98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } }
I20260808 22:27:13.779217 18093 catalog_manager.cc:5495] T b98873446d8a4f47a921298d4ba14fea P 9fbd96f2c37b4a6a89fdca1756ab8a98 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9fbd96f2c37b4a6a89fdca1756ab8a98 (127.5.11.195). New cstate: current_term: 1 leader_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } health_report { overall_health: HEALTHY } } }
Aug 08 22:27:13 dist-test-slave-7f01 krb5kdc[18029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228032, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.254@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM
I20260808 22:27:13.815339 18991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:13.805607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43883 (local address 127.0.0.1:41452)
0808 22:27:13.805930 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:13.805947 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:13.806076 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:13.806356 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:13.806359 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:13.806867 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:13.806967 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:13.806975 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.808096 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.808102 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:13.808666 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:13.808674 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.809147 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.809151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:13.809246 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:13.809882 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:13.809894 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:13.811557 (+ 1663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:13.813447 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:13.813458 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:13.813466 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:13.813817 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:13.814110 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:13.814114 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:13.814116 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:13.814223 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:13.814685 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:13.814692 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:13.814850 (+ 158us) client_negotiation.cc:769] Sending connection context
0808 22:27:13.814970 (+ 120us) client_negotiation.cc:240] Negotiation successful
0808 22:27:13.815123 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":5,"thread_start_us":116,"threads_started":1}
I20260808 22:27:13.816062 18992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:13.805664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41452 (local address 127.5.11.254:43883)
0808 22:27:13.805927 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:13.805936 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:13.806033 (+ 97us) server_negotiation.cc:407] Connection header received
0808 22:27:13.806195 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:13.806198 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:13.806245 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:13.806336 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:13.807095 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.807955 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.808791 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.809033 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.809144 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:13.811712 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:13.811715 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:13.811717 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:13.811802 (+ 85us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:13.813329 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:13.813917 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:13.813919 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:13.813920 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:13.814015 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:13.814322 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:13.814324 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:13.814326 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:13.814568 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:13.814675 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:13.815692 (+ 1017us) server_negotiation.cc:299] Negotiation successful
0808 22:27:13.815898 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":215,"thread_start_us":131,"threads_started":1}
I20260808 22:27:13.820214 18093 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:41452:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260808 22:27:13.820996 18093 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:27:13.826009 18546 tablet_service.cc:1467] Processing CreateTablet for tablet 993e0598966547afb9bfb92bbc8e49fe (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cff3a0b3c683436280679d1887320927]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 22:27:13.826134 18775 tablet_service.cc:1467] Processing CreateTablet for tablet 993e0598966547afb9bfb92bbc8e49fe (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cff3a0b3c683436280679d1887320927]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 22:27:13.826509 18775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 993e0598966547afb9bfb92bbc8e49fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:27:13.826520 18546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 993e0598966547afb9bfb92bbc8e49fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:27:13.829360 18941 tablet_bootstrap.cc:492] T 993e0598966547afb9bfb92bbc8e49fe P 7850c97cdd9d4c37b45acbccf1003368: Bootstrap starting.
I20260808 22:27:13.829458 18943 tablet_bootstrap.cc:492] T 993e0598966547afb9bfb92bbc8e49fe P 9fbd96f2c37b4a6a89fdca1756ab8a98: Bootstrap starting.
I20260808 22:27:13.829838 18327 tablet_service.cc:1467] Processing CreateTablet for tablet 993e0598966547afb9bfb92bbc8e49fe (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=cff3a0b3c683436280679d1887320927]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260808 22:27:13.830240 18327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 993e0598966547afb9bfb92bbc8e49fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:27:13.830315 18943 tablet_bootstrap.cc:654] T 993e0598966547afb9bfb92bbc8e49fe P 9fbd96f2c37b4a6a89fdca1756ab8a98: Neither blocks nor log segments found. Creating new log.
I20260808 22:27:13.830382 18941 tablet_bootstrap.cc:654] T 993e0598966547afb9bfb92bbc8e49fe P 7850c97cdd9d4c37b45acbccf1003368: Neither blocks nor log segments found. Creating new log.
I20260808 22:27:13.831905 18941 tablet_bootstrap.cc:492] T 993e0598966547afb9bfb92bbc8e49fe P 7850c97cdd9d4c37b45acbccf1003368: No bootstrap required, opened a new log
I20260808 22:27:13.832010 18941 ts_tablet_manager.cc:1397] T 993e0598966547afb9bfb92bbc8e49fe P 7850c97cdd9d4c37b45acbccf1003368: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 22:27:13.832481 18941 raft_consensus.cc:348] T 993e0598966547afb9bfb92bbc8e49fe P 7850c97cdd9d4c37b45acbccf1003368 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } }
I20260808 22:27:13.832614 18941 raft_consensus.cc:374] T 993e0598966547afb9bfb92bbc8e49fe P 7850c97cdd9d4c37b45acbccf1003368 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:27:13.832652 18941 raft_consensus.cc:729] T 993e0598966547afb9bfb92bbc8e49fe P 7850c97cdd9d4c37b45acbccf1003368 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7850c97cdd9d4c37b45acbccf1003368, State: Initialized, Role: FOLLOWER
I20260808 22:27:13.832799 18941 consensus_queue.cc:260] T 993e0598966547afb9bfb92bbc8e49fe P 7850c97cdd9d4c37b45acbccf1003368 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } }
I20260808 22:27:13.833099 18941 ts_tablet_manager.cc:1428] T 993e0598966547afb9bfb92bbc8e49fe P 7850c97cdd9d4c37b45acbccf1003368: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260808 22:27:13.833204 18943 tablet_bootstrap.cc:492] T 993e0598966547afb9bfb92bbc8e49fe P 9fbd96f2c37b4a6a89fdca1756ab8a98: No bootstrap required, opened a new log
I20260808 22:27:13.833298 18943 ts_tablet_manager.cc:1397] T 993e0598966547afb9bfb92bbc8e49fe P 9fbd96f2c37b4a6a89fdca1756ab8a98: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260808 22:27:13.833384 18948 tablet_replica.cc:439] T 993e0598966547afb9bfb92bbc8e49fe P 7850c97cdd9d4c37b45acbccf1003368: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } } }
I20260808 22:27:13.833534 18948 tablet_replica.cc:442] T 993e0598966547afb9bfb92bbc8e49fe P 7850c97cdd9d4c37b45acbccf1003368: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:27:13.833626 18942 tablet_bootstrap.cc:492] T 993e0598966547afb9bfb92bbc8e49fe P 638feafd83954241a681cb7e2ab8ec64: Bootstrap starting.
I20260808 22:27:13.833717 18943 raft_consensus.cc:348] T 993e0598966547afb9bfb92bbc8e49fe P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } }
I20260808 22:27:13.833827 18943 raft_consensus.cc:374] T 993e0598966547afb9bfb92bbc8e49fe P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:27:13.833863 18943 raft_consensus.cc:729] T 993e0598966547afb9bfb92bbc8e49fe P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fbd96f2c37b4a6a89fdca1756ab8a98, State: Initialized, Role: FOLLOWER
I20260808 22:27:13.833992 18943 consensus_queue.cc:260] T 993e0598966547afb9bfb92bbc8e49fe P 9fbd96f2c37b4a6a89fdca1756ab8a98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } }
I20260808 22:27:13.834285 18943 ts_tablet_manager.cc:1428] T 993e0598966547afb9bfb92bbc8e49fe P 9fbd96f2c37b4a6a89fdca1756ab8a98: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:27:13.834246 18950 tablet_replica.cc:439] T 993e0598966547afb9bfb92bbc8e49fe P 9fbd96f2c37b4a6a89fdca1756ab8a98: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } } }
I20260808 22:27:13.834372 18950 tablet_replica.cc:442] T 993e0598966547afb9bfb92bbc8e49fe P 9fbd96f2c37b4a6a89fdca1756ab8a98: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:27:13.834545 18942 tablet_bootstrap.cc:654] T 993e0598966547afb9bfb92bbc8e49fe P 638feafd83954241a681cb7e2ab8ec64: Neither blocks nor log segments found. Creating new log.
I20260808 22:27:13.836058 18942 tablet_bootstrap.cc:492] T 993e0598966547afb9bfb92bbc8e49fe P 638feafd83954241a681cb7e2ab8ec64: No bootstrap required, opened a new log
I20260808 22:27:13.836158 18942 ts_tablet_manager.cc:1397] T 993e0598966547afb9bfb92bbc8e49fe P 638feafd83954241a681cb7e2ab8ec64: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:27:13.836627 18942 raft_consensus.cc:348] T 993e0598966547afb9bfb92bbc8e49fe P 638feafd83954241a681cb7e2ab8ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } }
I20260808 22:27:13.836757 18942 raft_consensus.cc:374] T 993e0598966547afb9bfb92bbc8e49fe P 638feafd83954241a681cb7e2ab8ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:27:13.836794 18942 raft_consensus.cc:729] T 993e0598966547afb9bfb92bbc8e49fe P 638feafd83954241a681cb7e2ab8ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 638feafd83954241a681cb7e2ab8ec64, State: Initialized, Role: FOLLOWER
I20260808 22:27:13.836917 18942 consensus_queue.cc:260] T 993e0598966547afb9bfb92bbc8e49fe P 638feafd83954241a681cb7e2ab8ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } }
I20260808 22:27:13.837265 18942 ts_tablet_manager.cc:1428] T 993e0598966547afb9bfb92bbc8e49fe P 638feafd83954241a681cb7e2ab8ec64: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:27:13.837220 18949 tablet_replica.cc:439] T 993e0598966547afb9bfb92bbc8e49fe P 638feafd83954241a681cb7e2ab8ec64: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } } }
I20260808 22:27:13.837368 18949 tablet_replica.cc:442] T 993e0598966547afb9bfb92bbc8e49fe P 638feafd83954241a681cb7e2ab8ec64: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:27:13.965147 18948 raft_consensus.cc:482] T 993e0598966547afb9bfb92bbc8e49fe P 7850c97cdd9d4c37b45acbccf1003368 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:27:13.965334 18948 raft_consensus.cc:504] T 993e0598966547afb9bfb92bbc8e49fe P 7850c97cdd9d4c37b45acbccf1003368 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } }
I20260808 22:27:13.966602 18948 leader_election.cc:288] T 993e0598966547afb9bfb92bbc8e49fe P 7850c97cdd9d4c37b45acbccf1003368 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 638feafd83954241a681cb7e2ab8ec64 (127.5.11.193:43025), 9fbd96f2c37b4a6a89fdca1756ab8a98 (127.5.11.195:37603)
I20260808 22:27:13.971930 18940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:13.966962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:45533 (local address 127.5.11.193:43025)
0808 22:27:13.967125 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:13.967130 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:13.967369 (+ 239us) server_negotiation.cc:407] Connection header received
0808 22:27:13.967526 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:13.967529 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:13.967598 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:13.967665 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:27:13.968075 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.968893 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.970591 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.971226 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.971349 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:13.971404 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:13.971697 (+ 293us) server_negotiation.cc:299] Negotiation successful
0808 22:27:13.971800 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260808 22:27:13.971930 18938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:13.967463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:36085 (local address 127.5.11.195:37603)
0808 22:27:13.967599 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:13.967604 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:13.967821 (+ 217us) server_negotiation.cc:407] Connection header received
0808 22:27:13.967969 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:13.967972 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:13.968049 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:13.968132 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:27:13.968362 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.969063 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.970610 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.971226 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.971349 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:13.971404 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:13.971697 (+ 293us) server_negotiation.cc:299] Negotiation successful
0808 22:27:13.971796 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 22:27:13.971998 19001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:13.966868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:43025 (local address 127.5.11.194:45533)
0808 22:27:13.967214 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:13.967229 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:13.967398 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:13.967727 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:13.967754 (+ 27us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:13.967782 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:27:13.967953 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:13.967963 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.969005 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.969009 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:13.970461 (+ 1452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:13.970467 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.971353 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.971357 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:13.971536 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:13.971579 (+ 43us) client_negotiation.cc:769] Sending connection context
0808 22:27:13.971664 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 22:27:13.971795 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"mutex_wait_us":88,"spinlock_wait_cycles":3968,"thread_start_us":140,"threads_started":1}
I20260808 22:27:13.972133 19002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:13.967307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:37603 (local address 127.5.11.194:36085)
0808 22:27:13.967709 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:13.967722 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:13.967863 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:13.968148 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:13.968150 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:13.968167 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:27:13.968252 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:13.968256 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.969154 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.969157 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:13.970454 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:13.970467 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:13.971353 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:13.971357 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:13.971533 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:13.971579 (+ 46us) client_negotiation.cc:769] Sending connection context
0808 22:27:13.971664 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 22:27:13.971795 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":95,"spinlock_wait_cycles":3328,"thread_start_us":103,"threads_started":1}
I20260808 22:27:13.972844 18355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "993e0598966547afb9bfb92bbc8e49fe" candidate_uuid: "7850c97cdd9d4c37b45acbccf1003368" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "638feafd83954241a681cb7e2ab8ec64" is_pre_election: true
I20260808 22:27:13.972894 18814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "993e0598966547afb9bfb92bbc8e49fe" candidate_uuid: "7850c97cdd9d4c37b45acbccf1003368" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" is_pre_election: true
I20260808 22:27:13.972985 18355 raft_consensus.cc:2454] T 993e0598966547afb9bfb92bbc8e49fe P 638feafd83954241a681cb7e2ab8ec64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7850c97cdd9d4c37b45acbccf1003368 in term 0.
I20260808 22:27:13.973192 18814 raft_consensus.cc:2454] T 993e0598966547afb9bfb92bbc8e49fe P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7850c97cdd9d4c37b45acbccf1003368 in term 0.
I20260808 22:27:13.973435 18475 leader_election.cc:302] T 993e0598966547afb9bfb92bbc8e49fe P 7850c97cdd9d4c37b45acbccf1003368 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 638feafd83954241a681cb7e2ab8ec64, 7850c97cdd9d4c37b45acbccf1003368; no voters:
I20260808 22:27:13.973726 18948 raft_consensus.cc:2781] T 993e0598966547afb9bfb92bbc8e49fe P 7850c97cdd9d4c37b45acbccf1003368 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:27:13.973784 18948 raft_consensus.cc:482] T 993e0598966547afb9bfb92bbc8e49fe P 7850c97cdd9d4c37b45acbccf1003368 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:27:13.973812 18948 raft_consensus.cc:3037] T 993e0598966547afb9bfb92bbc8e49fe P 7850c97cdd9d4c37b45acbccf1003368 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:27:13.974836 18948 raft_consensus.cc:504] T 993e0598966547afb9bfb92bbc8e49fe P 7850c97cdd9d4c37b45acbccf1003368 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } }
I20260808 22:27:13.975188 18948 leader_election.cc:288] T 993e0598966547afb9bfb92bbc8e49fe P 7850c97cdd9d4c37b45acbccf1003368 [CANDIDATE]: Term 1 election: Requested vote from peers 638feafd83954241a681cb7e2ab8ec64 (127.5.11.193:43025), 9fbd96f2c37b4a6a89fdca1756ab8a98 (127.5.11.195:37603)
I20260808 22:27:13.975579 18355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "993e0598966547afb9bfb92bbc8e49fe" candidate_uuid: "7850c97cdd9d4c37b45acbccf1003368" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "638feafd83954241a681cb7e2ab8ec64"
I20260808 22:27:13.975615 18814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "993e0598966547afb9bfb92bbc8e49fe" candidate_uuid: "7850c97cdd9d4c37b45acbccf1003368" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98"
I20260808 22:27:13.975726 18355 raft_consensus.cc:3037] T 993e0598966547afb9bfb92bbc8e49fe P 638feafd83954241a681cb7e2ab8ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:27:13.975725 18814 raft_consensus.cc:3037] T 993e0598966547afb9bfb92bbc8e49fe P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:27:13.976835 18355 raft_consensus.cc:2454] T 993e0598966547afb9bfb92bbc8e49fe P 638feafd83954241a681cb7e2ab8ec64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7850c97cdd9d4c37b45acbccf1003368 in term 1.
I20260808 22:27:13.977097 18814 raft_consensus.cc:2454] T 993e0598966547afb9bfb92bbc8e49fe P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7850c97cdd9d4c37b45acbccf1003368 in term 1.
I20260808 22:27:13.977365 18475 leader_election.cc:302] T 993e0598966547afb9bfb92bbc8e49fe P 7850c97cdd9d4c37b45acbccf1003368 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7850c97cdd9d4c37b45acbccf1003368, 9fbd96f2c37b4a6a89fdca1756ab8a98; no voters:
I20260808 22:27:13.977726 18948 raft_consensus.cc:2781] T 993e0598966547afb9bfb92bbc8e49fe P 7850c97cdd9d4c37b45acbccf1003368 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:27:13.977942 18948 raft_consensus.cc:686] T 993e0598966547afb9bfb92bbc8e49fe P 7850c97cdd9d4c37b45acbccf1003368 [term 1 LEADER]: Becoming Leader. State: Replica: 7850c97cdd9d4c37b45acbccf1003368, State: Running, Role: LEADER
I20260808 22:27:13.978190 18948 consensus_queue.cc:237] T 993e0598966547afb9bfb92bbc8e49fe P 7850c97cdd9d4c37b45acbccf1003368 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } }
I20260808 22:27:13.979984 19003 tablet_replica.cc:439] T 993e0598966547afb9bfb92bbc8e49fe P 7850c97cdd9d4c37b45acbccf1003368: TxnStatusTablet state changed. Reason: New leader 7850c97cdd9d4c37b45acbccf1003368. Latest consensus state: current_term: 1 leader_uuid: "7850c97cdd9d4c37b45acbccf1003368" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } } }
I20260808 22:27:13.980100 19003 tablet_replica.cc:442] T 993e0598966547afb9bfb92bbc8e49fe P 7850c97cdd9d4c37b45acbccf1003368: This TxnStatusTablet replica's current role is: LEADER
I20260808 22:27:13.980610 19006 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260808 22:27:13.980763 19006 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260808 22:27:13.980697 18088 catalog_manager.cc:5495] T 993e0598966547afb9bfb92bbc8e49fe P 7850c97cdd9d4c37b45acbccf1003368 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7850c97cdd9d4c37b45acbccf1003368 (127.5.11.194). New cstate: current_term: 1 leader_uuid: "7850c97cdd9d4c37b45acbccf1003368" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } health_report { overall_health: UNKNOWN } } }
I20260808 22:27:14.024798 18934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:14.020649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51064 (local address 127.5.11.194:45435)
0808 22:27:14.020782 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:14.020788 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:14.020804 (+ 16us) server_negotiation.cc:407] Connection header received
0808 22:27:14.021024 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:14.021028 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:14.021105 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:14.021217 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:27:14.021461 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:14.022260 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:14.023096 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:14.023456 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:14.023560 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:14.023833 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:27:14.024391 (+ 558us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:27:14.024509 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:14.024591 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 22:27:14.024651 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 22:27:14.024796 18991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:14.020522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:45435 (local address 127.0.0.1:51064)
0808 22:27:14.020736 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:14.020759 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:14.020852 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:14.021235 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:14.021238 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:14.021245 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:27:14.021344 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:14.021348 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:14.022363 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:14.022365 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:14.022966 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:14.022974 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:14.023567 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:14.023569 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:14.023633 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:14.023682 (+ 49us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:27:14.024490 (+ 808us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:27:14.024505 (+ 15us) client_negotiation.cc:769] Sending connection context
0808 22:27:14.024571 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 22:27:14.024644 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83}
I20260808 22:27:14.028108 18355 raft_consensus.cc:1261] T 993e0598966547afb9bfb92bbc8e49fe P 638feafd83954241a681cb7e2ab8ec64 [term 1 FOLLOWER]: Refusing update from remote peer 7850c97cdd9d4c37b45acbccf1003368: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:27:14.028210 18814 raft_consensus.cc:1261] T 993e0598966547afb9bfb92bbc8e49fe P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 1 FOLLOWER]: Refusing update from remote peer 7850c97cdd9d4c37b45acbccf1003368: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:27:14.028642 18948 consensus_queue.cc:1035] T 993e0598966547afb9bfb92bbc8e49fe P 7850c97cdd9d4c37b45acbccf1003368 [LEADER]: Connected to new peer: Peer: permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:27:14.028831 19003 consensus_queue.cc:1035] T 993e0598966547afb9bfb92bbc8e49fe P 7850c97cdd9d4c37b45acbccf1003368 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:27:14.030530 18950 tablet_replica.cc:439] T 993e0598966547afb9bfb92bbc8e49fe P 9fbd96f2c37b4a6a89fdca1756ab8a98: TxnStatusTablet state changed. Reason: New leader 7850c97cdd9d4c37b45acbccf1003368. Latest consensus state: current_term: 1 leader_uuid: "7850c97cdd9d4c37b45acbccf1003368" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } } }
I20260808 22:27:14.030659 18950 tablet_replica.cc:442] T 993e0598966547afb9bfb92bbc8e49fe P 9fbd96f2c37b4a6a89fdca1756ab8a98: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:27:14.031255 18949 tablet_replica.cc:439] T 993e0598966547afb9bfb92bbc8e49fe P 638feafd83954241a681cb7e2ab8ec64: TxnStatusTablet state changed. Reason: New leader 7850c97cdd9d4c37b45acbccf1003368. Latest consensus state: current_term: 1 leader_uuid: "7850c97cdd9d4c37b45acbccf1003368" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } } }
I20260808 22:27:14.031370 18949 tablet_replica.cc:442] T 993e0598966547afb9bfb92bbc8e49fe P 638feafd83954241a681cb7e2ab8ec64: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:27:14.032742 19003 tablet_replica.cc:439] T 993e0598966547afb9bfb92bbc8e49fe P 7850c97cdd9d4c37b45acbccf1003368: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7850c97cdd9d4c37b45acbccf1003368" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } } }
I20260808 22:27:14.032842 19003 tablet_replica.cc:442] T 993e0598966547afb9bfb92bbc8e49fe P 7850c97cdd9d4c37b45acbccf1003368: This TxnStatusTablet replica's current role is: LEADER
I20260808 22:27:14.035095 18948 tablet_replica.cc:439] T 993e0598966547afb9bfb92bbc8e49fe P 7850c97cdd9d4c37b45acbccf1003368: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7850c97cdd9d4c37b45acbccf1003368" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } } }
I20260808 22:27:14.035207 18948 tablet_replica.cc:442] T 993e0598966547afb9bfb92bbc8e49fe P 7850c97cdd9d4c37b45acbccf1003368: This TxnStatusTablet replica's current role is: LEADER
I20260808 22:27:14.035243 18950 tablet_replica.cc:439] T 993e0598966547afb9bfb92bbc8e49fe P 9fbd96f2c37b4a6a89fdca1756ab8a98: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7850c97cdd9d4c37b45acbccf1003368" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } } }
I20260808 22:27:14.035360 18950 tablet_replica.cc:442] T 993e0598966547afb9bfb92bbc8e49fe P 9fbd96f2c37b4a6a89fdca1756ab8a98: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:27:14.036974 18958 mvcc.cc:205] Tried to move back new op lower bound from 7316390027373096960 to 7316390027177242624. Current Snapshot: MvccSnapshot[applied={T|T < 7316390027373096960}]
I20260808 22:27:14.037231 18949 tablet_replica.cc:439] T 993e0598966547afb9bfb92bbc8e49fe P 638feafd83954241a681cb7e2ab8ec64: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7850c97cdd9d4c37b45acbccf1003368" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } } }
I20260808 22:27:14.037343 18949 tablet_replica.cc:442] T 993e0598966547afb9bfb92bbc8e49fe P 638feafd83954241a681cb7e2ab8ec64: This TxnStatusTablet replica's current role is: FOLLOWER
I20260808 22:27:14.048712 19029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:14.045172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:45435 (local address 127.5.11.195:42563)
0808 22:27:14.045457 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:14.045479 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:14.045575 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:14.045810 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:14.045812 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:14.045818 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:27:14.045905 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:14.045909 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:14.046880 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:14.046882 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:14.047486 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:14.047494 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:14.048075 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:14.048078 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:14.048143 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:14.048187 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:27:14.048470 (+ 283us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:27:14.048475 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:27:14.048530 (+ 55us) client_negotiation.cc:240] Negotiation successful
0808 22:27:14.048599 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":100,"threads_started":1}
I20260808 22:27:14.048715 18934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:14.045267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:42563 (local address 127.5.11.194:45435)
0808 22:27:14.045380 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:14.045385 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:14.045558 (+ 173us) server_negotiation.cc:407] Connection header received
0808 22:27:14.045663 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:14.045666 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:14.045715 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:14.045794 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:27:14.046008 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:14.046760 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:14.047603 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:14.047963 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:14.048074 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:14.048312 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:27:14.048390 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:27:14.048460 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:14.048542 (+ 82us) server_negotiation.cc:299] Negotiation successful
0808 22:27:14.048591 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 22:27:14.055491 18355 raft_consensus.cc:1261] T b98873446d8a4f47a921298d4ba14fea P 638feafd83954241a681cb7e2ab8ec64 [term 1 FOLLOWER]: Refusing update from remote peer 9fbd96f2c37b4a6a89fdca1756ab8a98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:27:14.055577 18572 raft_consensus.cc:1261] T b98873446d8a4f47a921298d4ba14fea P 7850c97cdd9d4c37b45acbccf1003368 [term 1 FOLLOWER]: Refusing update from remote peer 9fbd96f2c37b4a6a89fdca1756ab8a98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:27:14.055984 18952 consensus_queue.cc:1035] T b98873446d8a4f47a921298d4ba14fea P 9fbd96f2c37b4a6a89fdca1756ab8a98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:27:14.056196 18950 consensus_queue.cc:1035] T b98873446d8a4f47a921298d4ba14fea P 9fbd96f2c37b4a6a89fdca1756ab8a98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:27:14.077577 18938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:14.073667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:54557 (local address 127.5.11.195:37603)
0808 22:27:14.073782 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:14.073787 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:14.074022 (+ 235us) server_negotiation.cc:407] Connection header received
0808 22:27:14.074113 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:14.074116 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:14.074166 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:14.074248 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:27:14.074505 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:14.075288 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:14.076402 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:14.076693 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:14.076806 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:14.077129 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:27:14.077207 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:27:14.077287 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:14.077375 (+ 88us) server_negotiation.cc:299] Negotiation successful
0808 22:27:14.077429 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 22:27:14.077592 19040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:14.073606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:37603 (local address 127.5.11.194:54557)
0808 22:27:14.073930 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:14.073943 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:14.074038 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:14.074260 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:14.074262 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:14.074279 (+ 17us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:27:14.074377 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:14.074381 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:14.075409 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:14.075412 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:14.076295 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:14.076305 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:14.076820 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:14.076824 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:14.076925 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:14.076971 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:27:14.077292 (+ 321us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:27:14.077302 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 22:27:14.077367 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 22:27:14.077444 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":113,"threads_started":1}
I20260808 22:27:14.096930 18093 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:27:14.097177 18093 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41442:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:27:14.099289 18093 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 258796a613b547bb87a6fa5fe06e6556: Sending DeleteTablet for 3 replicas of tablet b98873446d8a4f47a921298d4ba14fea
I20260808 22:27:14.099864 18327 tablet_service.cc:1514] Processing DeleteTablet for tablet b98873446d8a4f47a921298d4ba14fea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:27:14 UTC) from {username='oryx', principal='oryx/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:35954
I20260808 22:27:14.099961 18546 tablet_service.cc:1514] Processing DeleteTablet for tablet b98873446d8a4f47a921298d4ba14fea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:27:14 UTC) from {username='oryx', principal='oryx/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:51052
I20260808 22:27:14.100109 18969 tablet_replica.cc:330] T b98873446d8a4f47a921298d4ba14fea P 638feafd83954241a681cb7e2ab8ec64: stopping tablet replica
I20260808 22:27:14.100242 18969 raft_consensus.cc:2229] T b98873446d8a4f47a921298d4ba14fea P 638feafd83954241a681cb7e2ab8ec64 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:27:14.100250 18967 tablet_replica.cc:330] T b98873446d8a4f47a921298d4ba14fea P 7850c97cdd9d4c37b45acbccf1003368: stopping tablet replica
I20260808 22:27:14.100368 18967 raft_consensus.cc:2229] T b98873446d8a4f47a921298d4ba14fea P 7850c97cdd9d4c37b45acbccf1003368 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:27:14.100364 18969 raft_consensus.cc:2258] T b98873446d8a4f47a921298d4ba14fea P 638feafd83954241a681cb7e2ab8ec64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:27:14.100486 18967 raft_consensus.cc:2258] T b98873446d8a4f47a921298d4ba14fea P 7850c97cdd9d4c37b45acbccf1003368 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:27:14.100474 18775 tablet_service.cc:1514] Processing DeleteTablet for tablet b98873446d8a4f47a921298d4ba14fea with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:27:14 UTC) from {username='oryx', principal='oryx/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:47406
I20260808 22:27:14.100735 18968 tablet_replica.cc:330] T b98873446d8a4f47a921298d4ba14fea P 9fbd96f2c37b4a6a89fdca1756ab8a98: stopping tablet replica
I20260808 22:27:14.100857 18968 raft_consensus.cc:2229] T b98873446d8a4f47a921298d4ba14fea P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:27:14.101048 18968 raft_consensus.cc:2258] T b98873446d8a4f47a921298d4ba14fea P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:27:14.101375 18967 ts_tablet_manager.cc:1905] T b98873446d8a4f47a921298d4ba14fea P 7850c97cdd9d4c37b45acbccf1003368: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:27:14.101779 18968 ts_tablet_manager.cc:1905] T b98873446d8a4f47a921298d4ba14fea P 9fbd96f2c37b4a6a89fdca1756ab8a98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:27:14.104959 18967 ts_tablet_manager.cc:1918] T b98873446d8a4f47a921298d4ba14fea P 7850c97cdd9d4c37b45acbccf1003368: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:27:14.105199 18967 log.cc:1196] T b98873446d8a4f47a921298d4ba14fea P 7850c97cdd9d4c37b45acbccf1003368: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-1/wal/wals/b98873446d8a4f47a921298d4ba14fea
I20260808 22:27:14.105312 18968 ts_tablet_manager.cc:1918] T b98873446d8a4f47a921298d4ba14fea P 9fbd96f2c37b4a6a89fdca1756ab8a98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:27:14.105410 18968 log.cc:1196] T b98873446d8a4f47a921298d4ba14fea P 9fbd96f2c37b4a6a89fdca1756ab8a98: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-2/wal/wals/b98873446d8a4f47a921298d4ba14fea
I20260808 22:27:14.105717 18968 ts_tablet_manager.cc:1939] T b98873446d8a4f47a921298d4ba14fea P 9fbd96f2c37b4a6a89fdca1756ab8a98: Deleting consensus metadata
I20260808 22:27:14.105739 18967 ts_tablet_manager.cc:1939] T b98873446d8a4f47a921298d4ba14fea P 7850c97cdd9d4c37b45acbccf1003368: Deleting consensus metadata
I20260808 22:27:14.106565 18073 catalog_manager.cc:4841] TS 9fbd96f2c37b4a6a89fdca1756ab8a98 (127.5.11.195:37603): tablet b98873446d8a4f47a921298d4ba14fea (table test-table [id=440d887cb8604c48967b7ea8cce87206]) successfully deleted
I20260808 22:27:14.106750 18073 catalog_manager.cc:4841] TS 7850c97cdd9d4c37b45acbccf1003368 (127.5.11.194:45435): tablet b98873446d8a4f47a921298d4ba14fea (table test-table [id=440d887cb8604c48967b7ea8cce87206]) successfully deleted
I20260808 22:27:14.107568 18969 ts_tablet_manager.cc:1905] T b98873446d8a4f47a921298d4ba14fea P 638feafd83954241a681cb7e2ab8ec64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:27:14.110737 18969 ts_tablet_manager.cc:1918] T b98873446d8a4f47a921298d4ba14fea P 638feafd83954241a681cb7e2ab8ec64: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:27:14.110848 18969 log.cc:1196] T b98873446d8a4f47a921298d4ba14fea P 638feafd83954241a681cb7e2ab8ec64: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-0/wal/wals/b98873446d8a4f47a921298d4ba14fea
I20260808 22:27:14.111104 18969 ts_tablet_manager.cc:1939] T b98873446d8a4f47a921298d4ba14fea P 638feafd83954241a681cb7e2ab8ec64: Deleting consensus metadata
I20260808 22:27:14.111800 18073 catalog_manager.cc:4841] TS 638feafd83954241a681cb7e2ab8ec64 (127.5.11.193:43025): tablet b98873446d8a4f47a921298d4ba14fea (table test-table [id=440d887cb8604c48967b7ea8cce87206]) successfully deleted
Aug 08 22:27:14 dist-test-slave-7f01 krb5kdc[18029](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 08 22:27:14 dist-test-slave-7f01 krb5kdc[18029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:14 dist-test-slave-7f01 krb5kdc[18029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
W20260808 22:27:14.134074 18992 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:14.127090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41468 (local address 127.5.11.254:43883)
0808 22:27:14.127234 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:14.127239 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:14.127418 (+ 179us) server_negotiation.cc:407] Connection header received
0808 22:27:14.127591 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:14.127598 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:14.127649 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:14.127743 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:14.128561 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:14.129120 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:14.130066 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:14.130317 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:14.130433 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:14.133929 (+ 3496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41468: BlockingRecv error: recv got EOF from 127.0.0.1:41468 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 08 22:27:14 dist-test-slave-7f01 krb5kdc[18029](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228034, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM
I20260808 22:27:14.145052 18992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:14.135889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41476 (local address 127.5.11.254:43883)
0808 22:27:14.136043 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:14.136047 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:14.136156 (+ 109us) server_negotiation.cc:407] Connection header received
0808 22:27:14.136239 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:14.136242 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:14.136296 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:14.136392 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:14.136983 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:14.137766 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:14.138534 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:14.138860 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:14.138977 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:14.141568 (+ 2591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:14.141572 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:14.141575 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:14.141603 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:14.143255 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:14.143808 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:14.143812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:14.143813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:14.143896 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:14.144179 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:14.144181 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:14.144182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:14.144356 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:14.144465 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:14.144708 (+ 243us) server_negotiation.cc:299] Negotiation successful
0808 22:27:14.144835 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 22:27:14.147926 18093 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41476:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:27:14.148425 18093 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:27:14.153604 18327 tablet_service.cc:1467] Processing CreateTablet for tablet e89d23b5e7484a22b279bcaa819071c3 (DEFAULT_TABLE table=test-table [id=652f4fc85c40421d893d8990c1b5505b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:27:14.153714 18775 tablet_service.cc:1467] Processing CreateTablet for tablet e89d23b5e7484a22b279bcaa819071c3 (DEFAULT_TABLE table=test-table [id=652f4fc85c40421d893d8990c1b5505b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:27:14.153867 18546 tablet_service.cc:1467] Processing CreateTablet for tablet e89d23b5e7484a22b279bcaa819071c3 (DEFAULT_TABLE table=test-table [id=652f4fc85c40421d893d8990c1b5505b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:27:14.153942 18327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e89d23b5e7484a22b279bcaa819071c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:27:14.154022 18775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e89d23b5e7484a22b279bcaa819071c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:27:14.154160 18546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e89d23b5e7484a22b279bcaa819071c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:27:14.156706 18943 tablet_bootstrap.cc:492] T e89d23b5e7484a22b279bcaa819071c3 P 9fbd96f2c37b4a6a89fdca1756ab8a98: Bootstrap starting.
I20260808 22:27:14.156836 18942 tablet_bootstrap.cc:492] T e89d23b5e7484a22b279bcaa819071c3 P 638feafd83954241a681cb7e2ab8ec64: Bootstrap starting.
I20260808 22:27:14.157493 18943 tablet_bootstrap.cc:654] T e89d23b5e7484a22b279bcaa819071c3 P 9fbd96f2c37b4a6a89fdca1756ab8a98: Neither blocks nor log segments found. Creating new log.
I20260808 22:27:14.157874 18942 tablet_bootstrap.cc:654] T e89d23b5e7484a22b279bcaa819071c3 P 638feafd83954241a681cb7e2ab8ec64: Neither blocks nor log segments found. Creating new log.
I20260808 22:27:14.158114 18941 tablet_bootstrap.cc:492] T e89d23b5e7484a22b279bcaa819071c3 P 7850c97cdd9d4c37b45acbccf1003368: Bootstrap starting.
I20260808 22:27:14.158957 18941 tablet_bootstrap.cc:654] T e89d23b5e7484a22b279bcaa819071c3 P 7850c97cdd9d4c37b45acbccf1003368: Neither blocks nor log segments found. Creating new log.
I20260808 22:27:14.159101 18942 tablet_bootstrap.cc:492] T e89d23b5e7484a22b279bcaa819071c3 P 638feafd83954241a681cb7e2ab8ec64: No bootstrap required, opened a new log
I20260808 22:27:14.159192 18942 ts_tablet_manager.cc:1397] T e89d23b5e7484a22b279bcaa819071c3 P 638feafd83954241a681cb7e2ab8ec64: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260808 22:27:14.159695 18942 raft_consensus.cc:348] T e89d23b5e7484a22b279bcaa819071c3 P 638feafd83954241a681cb7e2ab8ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } }
I20260808 22:27:14.159879 18942 raft_consensus.cc:374] T e89d23b5e7484a22b279bcaa819071c3 P 638feafd83954241a681cb7e2ab8ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:27:14.159930 18942 raft_consensus.cc:729] T e89d23b5e7484a22b279bcaa819071c3 P 638feafd83954241a681cb7e2ab8ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 638feafd83954241a681cb7e2ab8ec64, State: Initialized, Role: FOLLOWER
I20260808 22:27:14.160105 18942 consensus_queue.cc:260] T e89d23b5e7484a22b279bcaa819071c3 P 638feafd83954241a681cb7e2ab8ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } }
I20260808 22:27:14.160426 18943 tablet_bootstrap.cc:492] T e89d23b5e7484a22b279bcaa819071c3 P 9fbd96f2c37b4a6a89fdca1756ab8a98: No bootstrap required, opened a new log
I20260808 22:27:14.160532 18943 ts_tablet_manager.cc:1397] T e89d23b5e7484a22b279bcaa819071c3 P 9fbd96f2c37b4a6a89fdca1756ab8a98: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 22:27:14.160817 18942 ts_tablet_manager.cc:1428] T e89d23b5e7484a22b279bcaa819071c3 P 638feafd83954241a681cb7e2ab8ec64: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260808 22:27:14.160946 18943 raft_consensus.cc:348] T e89d23b5e7484a22b279bcaa819071c3 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } }
I20260808 22:27:14.161062 18943 raft_consensus.cc:374] T e89d23b5e7484a22b279bcaa819071c3 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:27:14.161094 18943 raft_consensus.cc:729] T e89d23b5e7484a22b279bcaa819071c3 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fbd96f2c37b4a6a89fdca1756ab8a98, State: Initialized, Role: FOLLOWER
I20260808 22:27:14.161213 18943 consensus_queue.cc:260] T e89d23b5e7484a22b279bcaa819071c3 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } }
I20260808 22:27:14.161525 18943 ts_tablet_manager.cc:1428] T e89d23b5e7484a22b279bcaa819071c3 P 9fbd96f2c37b4a6a89fdca1756ab8a98: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:27:14.162063 18941 tablet_bootstrap.cc:492] T e89d23b5e7484a22b279bcaa819071c3 P 7850c97cdd9d4c37b45acbccf1003368: No bootstrap required, opened a new log
I20260808 22:27:14.162130 18941 ts_tablet_manager.cc:1397] T e89d23b5e7484a22b279bcaa819071c3 P 7850c97cdd9d4c37b45acbccf1003368: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260808 22:27:14.162554 18941 raft_consensus.cc:348] T e89d23b5e7484a22b279bcaa819071c3 P 7850c97cdd9d4c37b45acbccf1003368 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } }
I20260808 22:27:14.162678 18941 raft_consensus.cc:374] T e89d23b5e7484a22b279bcaa819071c3 P 7850c97cdd9d4c37b45acbccf1003368 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:27:14.162719 18941 raft_consensus.cc:729] T e89d23b5e7484a22b279bcaa819071c3 P 7850c97cdd9d4c37b45acbccf1003368 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7850c97cdd9d4c37b45acbccf1003368, State: Initialized, Role: FOLLOWER
I20260808 22:27:14.162863 18941 consensus_queue.cc:260] T e89d23b5e7484a22b279bcaa819071c3 P 7850c97cdd9d4c37b45acbccf1003368 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } }
I20260808 22:27:14.163169 18941 ts_tablet_manager.cc:1428] T e89d23b5e7484a22b279bcaa819071c3 P 7850c97cdd9d4c37b45acbccf1003368: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:27:14.232183 18950 raft_consensus.cc:482] T e89d23b5e7484a22b279bcaa819071c3 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:27:14.232357 18950 raft_consensus.cc:504] T e89d23b5e7484a22b279bcaa819071c3 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } }
I20260808 22:27:14.232774 18950 leader_election.cc:288] T e89d23b5e7484a22b279bcaa819071c3 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 638feafd83954241a681cb7e2ab8ec64 (127.5.11.193:43025), 7850c97cdd9d4c37b45acbccf1003368 (127.5.11.194:45435)
I20260808 22:27:14.233261 18355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e89d23b5e7484a22b279bcaa819071c3" candidate_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "638feafd83954241a681cb7e2ab8ec64" is_pre_election: true
I20260808 22:27:14.233317 18572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e89d23b5e7484a22b279bcaa819071c3" candidate_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7850c97cdd9d4c37b45acbccf1003368" is_pre_election: true
I20260808 22:27:14.233430 18355 raft_consensus.cc:2454] T e89d23b5e7484a22b279bcaa819071c3 P 638feafd83954241a681cb7e2ab8ec64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9fbd96f2c37b4a6a89fdca1756ab8a98 in term 0.
I20260808 22:27:14.233487 18572 raft_consensus.cc:2454] T e89d23b5e7484a22b279bcaa819071c3 P 7850c97cdd9d4c37b45acbccf1003368 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9fbd96f2c37b4a6a89fdca1756ab8a98 in term 0.
I20260808 22:27:14.233771 18711 leader_election.cc:302] T e89d23b5e7484a22b279bcaa819071c3 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 638feafd83954241a681cb7e2ab8ec64, 9fbd96f2c37b4a6a89fdca1756ab8a98; no voters:
I20260808 22:27:14.233965 18950 raft_consensus.cc:2781] T e89d23b5e7484a22b279bcaa819071c3 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:27:14.234025 18950 raft_consensus.cc:482] T e89d23b5e7484a22b279bcaa819071c3 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:27:14.234051 18950 raft_consensus.cc:3037] T e89d23b5e7484a22b279bcaa819071c3 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:27:14.235083 18950 raft_consensus.cc:504] T e89d23b5e7484a22b279bcaa819071c3 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } }
I20260808 22:27:14.235407 18950 leader_election.cc:288] T e89d23b5e7484a22b279bcaa819071c3 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [CANDIDATE]: Term 1 election: Requested vote from peers 638feafd83954241a681cb7e2ab8ec64 (127.5.11.193:43025), 7850c97cdd9d4c37b45acbccf1003368 (127.5.11.194:45435)
I20260808 22:27:14.235841 18572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e89d23b5e7484a22b279bcaa819071c3" candidate_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7850c97cdd9d4c37b45acbccf1003368"
I20260808 22:27:14.235852 18355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e89d23b5e7484a22b279bcaa819071c3" candidate_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "638feafd83954241a681cb7e2ab8ec64"
I20260808 22:27:14.235941 18572 raft_consensus.cc:3037] T e89d23b5e7484a22b279bcaa819071c3 P 7850c97cdd9d4c37b45acbccf1003368 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:27:14.235984 18355 raft_consensus.cc:3037] T e89d23b5e7484a22b279bcaa819071c3 P 638feafd83954241a681cb7e2ab8ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:27:14.236866 18572 raft_consensus.cc:2454] T e89d23b5e7484a22b279bcaa819071c3 P 7850c97cdd9d4c37b45acbccf1003368 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fbd96f2c37b4a6a89fdca1756ab8a98 in term 1.
I20260808 22:27:14.236939 18355 raft_consensus.cc:2454] T e89d23b5e7484a22b279bcaa819071c3 P 638feafd83954241a681cb7e2ab8ec64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9fbd96f2c37b4a6a89fdca1756ab8a98 in term 1.
I20260808 22:27:14.237190 18711 leader_election.cc:302] T e89d23b5e7484a22b279bcaa819071c3 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7850c97cdd9d4c37b45acbccf1003368, 9fbd96f2c37b4a6a89fdca1756ab8a98; no voters:
I20260808 22:27:14.237388 18950 raft_consensus.cc:2781] T e89d23b5e7484a22b279bcaa819071c3 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:27:14.237533 18950 raft_consensus.cc:686] T e89d23b5e7484a22b279bcaa819071c3 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 1 LEADER]: Becoming Leader. State: Replica: 9fbd96f2c37b4a6a89fdca1756ab8a98, State: Running, Role: LEADER
I20260808 22:27:14.237682 18950 consensus_queue.cc:237] T e89d23b5e7484a22b279bcaa819071c3 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } }
I20260808 22:27:14.239328 18093 catalog_manager.cc:5495] T e89d23b5e7484a22b279bcaa819071c3 P 9fbd96f2c37b4a6a89fdca1756ab8a98 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9fbd96f2c37b4a6a89fdca1756ab8a98 (127.5.11.195). New cstate: current_term: 1 leader_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } health_report { overall_health: HEALTHY } } }
I20260808 22:27:14.266256 18938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:14.263559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47422 (local address 127.5.11.195:37603)
0808 22:27:14.263695 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:14.263698 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:14.263711 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:27:14.263769 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:14.263771 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:14.263821 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:14.263885 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:27:14.264138 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:14.264632 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:14.265412 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:14.265619 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:14.265692 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:14.265914 (+ 222us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:27:14.265978 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:27:14.266030 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:14.266111 (+ 81us) server_negotiation.cc:299] Negotiation successful
0808 22:27:14.266154 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 22:27:14.268419 18355 raft_consensus.cc:1261] T e89d23b5e7484a22b279bcaa819071c3 P 638feafd83954241a681cb7e2ab8ec64 [term 1 FOLLOWER]: Refusing update from remote peer 9fbd96f2c37b4a6a89fdca1756ab8a98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:27:14.268426 18572 raft_consensus.cc:1261] T e89d23b5e7484a22b279bcaa819071c3 P 7850c97cdd9d4c37b45acbccf1003368 [term 1 FOLLOWER]: Refusing update from remote peer 9fbd96f2c37b4a6a89fdca1756ab8a98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:27:14.268908 18950 consensus_queue.cc:1035] T e89d23b5e7484a22b279bcaa819071c3 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:27:14.269063 18952 consensus_queue.cc:1035] T e89d23b5e7484a22b279bcaa819071c3 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:27:14.276420 18093 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:27:14.276590 18093 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:27:14.278465 18093 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 258796a613b547bb87a6fa5fe06e6556: Sending DeleteTablet for 3 replicas of tablet e89d23b5e7484a22b279bcaa819071c3
I20260808 22:27:14.279013 18327 tablet_service.cc:1514] Processing DeleteTablet for tablet e89d23b5e7484a22b279bcaa819071c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:27:14 UTC) from {username='oryx', principal='oryx/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:35954
I20260808 22:27:14.279111 18546 tablet_service.cc:1514] Processing DeleteTablet for tablet e89d23b5e7484a22b279bcaa819071c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:27:14 UTC) from {username='oryx', principal='oryx/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:51052
I20260808 22:27:14.279222 18969 tablet_replica.cc:330] T e89d23b5e7484a22b279bcaa819071c3 P 638feafd83954241a681cb7e2ab8ec64: stopping tablet replica
I20260808 22:27:14.279295 18967 tablet_replica.cc:330] T e89d23b5e7484a22b279bcaa819071c3 P 7850c97cdd9d4c37b45acbccf1003368: stopping tablet replica
I20260808 22:27:14.279333 18969 raft_consensus.cc:2229] T e89d23b5e7484a22b279bcaa819071c3 P 638feafd83954241a681cb7e2ab8ec64 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:27:14.279418 18967 raft_consensus.cc:2229] T e89d23b5e7484a22b279bcaa819071c3 P 7850c97cdd9d4c37b45acbccf1003368 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:27:14.279474 18969 raft_consensus.cc:2258] T e89d23b5e7484a22b279bcaa819071c3 P 638feafd83954241a681cb7e2ab8ec64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:27:14.279574 18967 raft_consensus.cc:2258] T e89d23b5e7484a22b279bcaa819071c3 P 7850c97cdd9d4c37b45acbccf1003368 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:27:14.279850 18775 tablet_service.cc:1514] Processing DeleteTablet for tablet e89d23b5e7484a22b279bcaa819071c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:27:14 UTC) from {username='oryx', principal='oryx/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:47406
I20260808 22:27:14.280019 18968 tablet_replica.cc:330] T e89d23b5e7484a22b279bcaa819071c3 P 9fbd96f2c37b4a6a89fdca1756ab8a98: stopping tablet replica
I20260808 22:27:14.280146 18968 raft_consensus.cc:2229] T e89d23b5e7484a22b279bcaa819071c3 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:27:14.280344 18968 raft_consensus.cc:2258] T e89d23b5e7484a22b279bcaa819071c3 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:27:14.280354 18969 ts_tablet_manager.cc:1905] T e89d23b5e7484a22b279bcaa819071c3 P 638feafd83954241a681cb7e2ab8ec64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:27:14.280365 18967 ts_tablet_manager.cc:1905] T e89d23b5e7484a22b279bcaa819071c3 P 7850c97cdd9d4c37b45acbccf1003368: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:27:14.282403 18969 ts_tablet_manager.cc:1918] T e89d23b5e7484a22b279bcaa819071c3 P 638feafd83954241a681cb7e2ab8ec64: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:27:14.282470 18969 log.cc:1196] T e89d23b5e7484a22b279bcaa819071c3 P 638feafd83954241a681cb7e2ab8ec64: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-0/wal/wals/e89d23b5e7484a22b279bcaa819071c3
I20260808 22:27:14.282514 18967 ts_tablet_manager.cc:1918] T e89d23b5e7484a22b279bcaa819071c3 P 7850c97cdd9d4c37b45acbccf1003368: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:27:14.282629 18967 log.cc:1196] T e89d23b5e7484a22b279bcaa819071c3 P 7850c97cdd9d4c37b45acbccf1003368: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-1/wal/wals/e89d23b5e7484a22b279bcaa819071c3
I20260808 22:27:14.282753 18969 ts_tablet_manager.cc:1939] T e89d23b5e7484a22b279bcaa819071c3 P 638feafd83954241a681cb7e2ab8ec64: Deleting consensus metadata
I20260808 22:27:14.282919 18967 ts_tablet_manager.cc:1939] T e89d23b5e7484a22b279bcaa819071c3 P 7850c97cdd9d4c37b45acbccf1003368: Deleting consensus metadata
I20260808 22:27:14.283483 18073 catalog_manager.cc:4841] TS 638feafd83954241a681cb7e2ab8ec64 (127.5.11.193:43025): tablet e89d23b5e7484a22b279bcaa819071c3 (table test-table [id=652f4fc85c40421d893d8990c1b5505b]) successfully deleted
I20260808 22:27:14.283677 18073 catalog_manager.cc:4841] TS 7850c97cdd9d4c37b45acbccf1003368 (127.5.11.194:45435): tablet e89d23b5e7484a22b279bcaa819071c3 (table test-table [id=652f4fc85c40421d893d8990c1b5505b]) successfully deleted
I20260808 22:27:14.283710 18093 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41476:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260808 22:27:14.284106 18968 ts_tablet_manager.cc:1905] T e89d23b5e7484a22b279bcaa819071c3 P 9fbd96f2c37b4a6a89fdca1756ab8a98: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260808 22:27:14.284183 18093 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:27:14.286494 18968 ts_tablet_manager.cc:1918] T e89d23b5e7484a22b279bcaa819071c3 P 9fbd96f2c37b4a6a89fdca1756ab8a98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:27:14.286578 18968 log.cc:1196] T e89d23b5e7484a22b279bcaa819071c3 P 9fbd96f2c37b4a6a89fdca1756ab8a98: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-2/wal/wals/e89d23b5e7484a22b279bcaa819071c3
I20260808 22:27:14.286868 18968 ts_tablet_manager.cc:1939] T e89d23b5e7484a22b279bcaa819071c3 P 9fbd96f2c37b4a6a89fdca1756ab8a98: Deleting consensus metadata
I20260808 22:27:14.287649 18073 catalog_manager.cc:4841] TS 9fbd96f2c37b4a6a89fdca1756ab8a98 (127.5.11.195:37603): tablet e89d23b5e7484a22b279bcaa819071c3 (table test-table [id=652f4fc85c40421d893d8990c1b5505b]) successfully deleted
I20260808 22:27:14.288785 18327 tablet_service.cc:1467] Processing CreateTablet for tablet 41945de770d54b25808aa55087fb5555 (DEFAULT_TABLE table=test-table [id=0c9edcddc9fd4f8e9c9b57da9aca466d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:27:14.288857 18775 tablet_service.cc:1467] Processing CreateTablet for tablet 41945de770d54b25808aa55087fb5555 (DEFAULT_TABLE table=test-table [id=0c9edcddc9fd4f8e9c9b57da9aca466d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:27:14.289083 18327 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41945de770d54b25808aa55087fb5555. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:27:14.289150 18775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41945de770d54b25808aa55087fb5555. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:27:14.290822 18943 tablet_bootstrap.cc:492] T 41945de770d54b25808aa55087fb5555 P 9fbd96f2c37b4a6a89fdca1756ab8a98: Bootstrap starting.
I20260808 22:27:14.290963 18942 tablet_bootstrap.cc:492] T 41945de770d54b25808aa55087fb5555 P 638feafd83954241a681cb7e2ab8ec64: Bootstrap starting.
I20260808 22:27:14.291702 18943 tablet_bootstrap.cc:654] T 41945de770d54b25808aa55087fb5555 P 9fbd96f2c37b4a6a89fdca1756ab8a98: Neither blocks nor log segments found. Creating new log.
I20260808 22:27:14.292413 18942 tablet_bootstrap.cc:654] T 41945de770d54b25808aa55087fb5555 P 638feafd83954241a681cb7e2ab8ec64: Neither blocks nor log segments found. Creating new log.
I20260808 22:27:14.292898 18943 tablet_bootstrap.cc:492] T 41945de770d54b25808aa55087fb5555 P 9fbd96f2c37b4a6a89fdca1756ab8a98: No bootstrap required, opened a new log
I20260808 22:27:14.293002 18943 ts_tablet_manager.cc:1397] T 41945de770d54b25808aa55087fb5555 P 9fbd96f2c37b4a6a89fdca1756ab8a98: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260808 22:27:14.293221 18546 tablet_service.cc:1467] Processing CreateTablet for tablet 41945de770d54b25808aa55087fb5555 (DEFAULT_TABLE table=test-table [id=0c9edcddc9fd4f8e9c9b57da9aca466d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:27:14.293411 18943 raft_consensus.cc:348] T 41945de770d54b25808aa55087fb5555 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } }
I20260808 22:27:14.293503 18546 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41945de770d54b25808aa55087fb5555. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:27:14.293547 18943 raft_consensus.cc:374] T 41945de770d54b25808aa55087fb5555 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:27:14.293581 18943 raft_consensus.cc:729] T 41945de770d54b25808aa55087fb5555 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fbd96f2c37b4a6a89fdca1756ab8a98, State: Initialized, Role: FOLLOWER
I20260808 22:27:14.293660 18942 tablet_bootstrap.cc:492] T 41945de770d54b25808aa55087fb5555 P 638feafd83954241a681cb7e2ab8ec64: No bootstrap required, opened a new log
I20260808 22:27:14.293751 18942 ts_tablet_manager.cc:1397] T 41945de770d54b25808aa55087fb5555 P 638feafd83954241a681cb7e2ab8ec64: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260808 22:27:14.293712 18943 consensus_queue.cc:260] T 41945de770d54b25808aa55087fb5555 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } }
I20260808 22:27:14.293996 18943 ts_tablet_manager.cc:1428] T 41945de770d54b25808aa55087fb5555 P 9fbd96f2c37b4a6a89fdca1756ab8a98: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 22:27:14.294220 18942 raft_consensus.cc:348] T 41945de770d54b25808aa55087fb5555 P 638feafd83954241a681cb7e2ab8ec64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } }
I20260808 22:27:14.294337 18942 raft_consensus.cc:374] T 41945de770d54b25808aa55087fb5555 P 638feafd83954241a681cb7e2ab8ec64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:27:14.294373 18942 raft_consensus.cc:729] T 41945de770d54b25808aa55087fb5555 P 638feafd83954241a681cb7e2ab8ec64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 638feafd83954241a681cb7e2ab8ec64, State: Initialized, Role: FOLLOWER
I20260808 22:27:14.294493 18942 consensus_queue.cc:260] T 41945de770d54b25808aa55087fb5555 P 638feafd83954241a681cb7e2ab8ec64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } }
I20260808 22:27:14.294852 18942 ts_tablet_manager.cc:1428] T 41945de770d54b25808aa55087fb5555 P 638feafd83954241a681cb7e2ab8ec64: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260808 22:27:14.295981 18941 tablet_bootstrap.cc:492] T 41945de770d54b25808aa55087fb5555 P 7850c97cdd9d4c37b45acbccf1003368: Bootstrap starting.
I20260808 22:27:14.296862 18941 tablet_bootstrap.cc:654] T 41945de770d54b25808aa55087fb5555 P 7850c97cdd9d4c37b45acbccf1003368: Neither blocks nor log segments found. Creating new log.
I20260808 22:27:14.298022 18941 tablet_bootstrap.cc:492] T 41945de770d54b25808aa55087fb5555 P 7850c97cdd9d4c37b45acbccf1003368: No bootstrap required, opened a new log
I20260808 22:27:14.298122 18941 ts_tablet_manager.cc:1397] T 41945de770d54b25808aa55087fb5555 P 7850c97cdd9d4c37b45acbccf1003368: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260808 22:27:14.298548 18941 raft_consensus.cc:348] T 41945de770d54b25808aa55087fb5555 P 7850c97cdd9d4c37b45acbccf1003368 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } }
I20260808 22:27:14.298686 18941 raft_consensus.cc:374] T 41945de770d54b25808aa55087fb5555 P 7850c97cdd9d4c37b45acbccf1003368 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:27:14.298723 18941 raft_consensus.cc:729] T 41945de770d54b25808aa55087fb5555 P 7850c97cdd9d4c37b45acbccf1003368 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7850c97cdd9d4c37b45acbccf1003368, State: Initialized, Role: FOLLOWER
I20260808 22:27:14.298859 18941 consensus_queue.cc:260] T 41945de770d54b25808aa55087fb5555 P 7850c97cdd9d4c37b45acbccf1003368 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } }
I20260808 22:27:14.299155 18941 ts_tablet_manager.cc:1428] T 41945de770d54b25808aa55087fb5555 P 7850c97cdd9d4c37b45acbccf1003368: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260808 22:27:14.384485 19003 raft_consensus.cc:482] T 41945de770d54b25808aa55087fb5555 P 7850c97cdd9d4c37b45acbccf1003368 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:27:14.384645 19003 raft_consensus.cc:504] T 41945de770d54b25808aa55087fb5555 P 7850c97cdd9d4c37b45acbccf1003368 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } }
I20260808 22:27:14.385155 19003 leader_election.cc:288] T 41945de770d54b25808aa55087fb5555 P 7850c97cdd9d4c37b45acbccf1003368 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 638feafd83954241a681cb7e2ab8ec64 (127.5.11.193:43025), 9fbd96f2c37b4a6a89fdca1756ab8a98 (127.5.11.195:37603)
I20260808 22:27:14.385738 18814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41945de770d54b25808aa55087fb5555" candidate_uuid: "7850c97cdd9d4c37b45acbccf1003368" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" is_pre_election: true
I20260808 22:27:14.385738 18355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41945de770d54b25808aa55087fb5555" candidate_uuid: "7850c97cdd9d4c37b45acbccf1003368" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "638feafd83954241a681cb7e2ab8ec64" is_pre_election: true
I20260808 22:27:14.385921 18814 raft_consensus.cc:2454] T 41945de770d54b25808aa55087fb5555 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7850c97cdd9d4c37b45acbccf1003368 in term 0.
I20260808 22:27:14.385919 18355 raft_consensus.cc:2454] T 41945de770d54b25808aa55087fb5555 P 638feafd83954241a681cb7e2ab8ec64 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7850c97cdd9d4c37b45acbccf1003368 in term 0.
I20260808 22:27:14.386271 18475 leader_election.cc:302] T 41945de770d54b25808aa55087fb5555 P 7850c97cdd9d4c37b45acbccf1003368 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 638feafd83954241a681cb7e2ab8ec64, 7850c97cdd9d4c37b45acbccf1003368; no voters:
I20260808 22:27:14.386505 19003 raft_consensus.cc:2781] T 41945de770d54b25808aa55087fb5555 P 7850c97cdd9d4c37b45acbccf1003368 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:27:14.386569 19003 raft_consensus.cc:482] T 41945de770d54b25808aa55087fb5555 P 7850c97cdd9d4c37b45acbccf1003368 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:27:14.386596 19003 raft_consensus.cc:3037] T 41945de770d54b25808aa55087fb5555 P 7850c97cdd9d4c37b45acbccf1003368 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:27:14.387697 19003 raft_consensus.cc:504] T 41945de770d54b25808aa55087fb5555 P 7850c97cdd9d4c37b45acbccf1003368 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } }
I20260808 22:27:14.388124 19003 leader_election.cc:288] T 41945de770d54b25808aa55087fb5555 P 7850c97cdd9d4c37b45acbccf1003368 [CANDIDATE]: Term 1 election: Requested vote from peers 638feafd83954241a681cb7e2ab8ec64 (127.5.11.193:43025), 9fbd96f2c37b4a6a89fdca1756ab8a98 (127.5.11.195:37603)
I20260808 22:27:14.388517 18355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41945de770d54b25808aa55087fb5555" candidate_uuid: "7850c97cdd9d4c37b45acbccf1003368" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "638feafd83954241a681cb7e2ab8ec64"
I20260808 22:27:14.388579 18814 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41945de770d54b25808aa55087fb5555" candidate_uuid: "7850c97cdd9d4c37b45acbccf1003368" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98"
I20260808 22:27:14.388664 18355 raft_consensus.cc:3037] T 41945de770d54b25808aa55087fb5555 P 638feafd83954241a681cb7e2ab8ec64 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:27:14.388731 18814 raft_consensus.cc:3037] T 41945de770d54b25808aa55087fb5555 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:27:14.389637 18355 raft_consensus.cc:2454] T 41945de770d54b25808aa55087fb5555 P 638feafd83954241a681cb7e2ab8ec64 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7850c97cdd9d4c37b45acbccf1003368 in term 1.
I20260808 22:27:14.389684 18814 raft_consensus.cc:2454] T 41945de770d54b25808aa55087fb5555 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7850c97cdd9d4c37b45acbccf1003368 in term 1.
I20260808 22:27:14.389945 18475 leader_election.cc:302] T 41945de770d54b25808aa55087fb5555 P 7850c97cdd9d4c37b45acbccf1003368 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 638feafd83954241a681cb7e2ab8ec64, 7850c97cdd9d4c37b45acbccf1003368; no voters:
I20260808 22:27:14.390141 19003 raft_consensus.cc:2781] T 41945de770d54b25808aa55087fb5555 P 7850c97cdd9d4c37b45acbccf1003368 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:27:14.390241 19003 raft_consensus.cc:686] T 41945de770d54b25808aa55087fb5555 P 7850c97cdd9d4c37b45acbccf1003368 [term 1 LEADER]: Becoming Leader. State: Replica: 7850c97cdd9d4c37b45acbccf1003368, State: Running, Role: LEADER
I20260808 22:27:14.390408 19003 consensus_queue.cc:237] T 41945de770d54b25808aa55087fb5555 P 7850c97cdd9d4c37b45acbccf1003368 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } }
I20260808 22:27:14.392243 18088 catalog_manager.cc:5495] T 41945de770d54b25808aa55087fb5555 P 7850c97cdd9d4c37b45acbccf1003368 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7850c97cdd9d4c37b45acbccf1003368 (127.5.11.194). New cstate: current_term: 1 leader_uuid: "7850c97cdd9d4c37b45acbccf1003368" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7850c97cdd9d4c37b45acbccf1003368" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 45435 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 } health_report { overall_health: UNKNOWN } } }
I20260808 22:27:14.428745 18934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:14.425553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51074 (local address 127.5.11.194:45435)
0808 22:27:14.425679 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:14.425686 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:14.425764 (+ 78us) server_negotiation.cc:407] Connection header received
0808 22:27:14.425918 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:14.425921 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:14.425976 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:14.426061 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:27:14.426317 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:14.426844 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:14.427676 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:14.427920 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:14.428035 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:14.428293 (+ 258us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:27:14.428374 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:27:14.428456 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:14.428551 (+ 95us) server_negotiation.cc:299] Negotiation successful
0808 22:27:14.428601 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 22:27:14.437033 18934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:14.434398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:48999 (local address 127.5.11.194:45435)
0808 22:27:14.434508 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:14.434512 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:14.434554 (+ 42us) server_negotiation.cc:407] Connection header received
0808 22:27:14.434651 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:14.434654 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:14.434692 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:14.434753 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:27:14.434976 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:14.435436 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:14.436185 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:14.436381 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:14.436456 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:14.436651 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:27:14.436707 (+ 56us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:27:14.436759 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:14.436862 (+ 103us) server_negotiation.cc:299] Negotiation successful
0808 22:27:14.436922 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 22:27:14.437033 19040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:14.434345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:45435 (local address 127.5.11.194:48999)
0808 22:27:14.434482 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:14.434503 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:14.434574 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:14.434780 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:14.434782 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:14.434788 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0808 22:27:14.434865 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:14.434869 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:14.435522 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:14.435525 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:14.436092 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:14.436100 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:14.436477 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:14.436479 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:14.436536 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:14.436559 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0808 22:27:14.436780 (+ 221us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0808 22:27:14.436785 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:27:14.436833 (+ 48us) client_negotiation.cc:240] Negotiation successful
0808 22:27:14.436891 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 22:27:14.442505 18814 raft_consensus.cc:1261] T 41945de770d54b25808aa55087fb5555 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 1 FOLLOWER]: Refusing update from remote peer 7850c97cdd9d4c37b45acbccf1003368: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:27:14.442523 18355 raft_consensus.cc:1261] T 41945de770d54b25808aa55087fb5555 P 638feafd83954241a681cb7e2ab8ec64 [term 1 FOLLOWER]: Refusing update from remote peer 7850c97cdd9d4c37b45acbccf1003368: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:27:14.442921 19003 consensus_queue.cc:1035] T 41945de770d54b25808aa55087fb5555 P 7850c97cdd9d4c37b45acbccf1003368 [LEADER]: Connected to new peer: Peer: permanent_uuid: "638feafd83954241a681cb7e2ab8ec64" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43025 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:27:14.443125 18948 consensus_queue.cc:1035] T 41945de770d54b25808aa55087fb5555 P 7850c97cdd9d4c37b45acbccf1003368 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fbd96f2c37b4a6a89fdca1756ab8a98" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:27:14.477540 18092 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:27:14.477722 18092 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41476:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:27:14.479830 18092 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 258796a613b547bb87a6fa5fe06e6556: Sending DeleteTablet for 3 replicas of tablet 41945de770d54b25808aa55087fb5555
I20260808 22:27:14.480365 18327 tablet_service.cc:1514] Processing DeleteTablet for tablet 41945de770d54b25808aa55087fb5555 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:27:14 UTC) from {username='oryx', principal='oryx/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:35954
I20260808 22:27:14.480419 18546 tablet_service.cc:1514] Processing DeleteTablet for tablet 41945de770d54b25808aa55087fb5555 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:27:14 UTC) from {username='oryx', principal='oryx/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:51052
I20260808 22:27:14.480466 18775 tablet_service.cc:1514] Processing DeleteTablet for tablet 41945de770d54b25808aa55087fb5555 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:27:14 UTC) from {username='oryx', principal='oryx/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:47406
I20260808 22:27:14.480572 18969 tablet_replica.cc:330] T 41945de770d54b25808aa55087fb5555 P 638feafd83954241a681cb7e2ab8ec64: stopping tablet replica
I20260808 22:27:14.480583 18967 tablet_replica.cc:330] T 41945de770d54b25808aa55087fb5555 P 7850c97cdd9d4c37b45acbccf1003368: stopping tablet replica
I20260808 22:27:14.480688 18969 raft_consensus.cc:2229] T 41945de770d54b25808aa55087fb5555 P 638feafd83954241a681cb7e2ab8ec64 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:27:14.480688 18967 raft_consensus.cc:2229] T 41945de770d54b25808aa55087fb5555 P 7850c97cdd9d4c37b45acbccf1003368 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:27:14.480792 18969 raft_consensus.cc:2258] T 41945de770d54b25808aa55087fb5555 P 638feafd83954241a681cb7e2ab8ec64 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:27:14.480973 18967 raft_consensus.cc:2258] T 41945de770d54b25808aa55087fb5555 P 7850c97cdd9d4c37b45acbccf1003368 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:27:14.481673 18967 ts_tablet_manager.cc:1905] T 41945de770d54b25808aa55087fb5555 P 7850c97cdd9d4c37b45acbccf1003368: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:27:14.483855 18968 tablet_replica.cc:330] T 41945de770d54b25808aa55087fb5555 P 9fbd96f2c37b4a6a89fdca1756ab8a98: stopping tablet replica
I20260808 22:27:14.483975 18968 raft_consensus.cc:2229] T 41945de770d54b25808aa55087fb5555 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:27:14.484109 18968 raft_consensus.cc:2258] T 41945de770d54b25808aa55087fb5555 P 9fbd96f2c37b4a6a89fdca1756ab8a98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:27:14.484431 18967 ts_tablet_manager.cc:1918] T 41945de770d54b25808aa55087fb5555 P 7850c97cdd9d4c37b45acbccf1003368: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:27:14.484510 18967 log.cc:1196] T 41945de770d54b25808aa55087fb5555 P 7850c97cdd9d4c37b45acbccf1003368: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-1/wal/wals/41945de770d54b25808aa55087fb5555
I20260808 22:27:14.484803 18967 ts_tablet_manager.cc:1939] T 41945de770d54b25808aa55087fb5555 P 7850c97cdd9d4c37b45acbccf1003368: Deleting consensus metadata
I20260808 22:27:14.485527 18968 ts_tablet_manager.cc:1905] T 41945de770d54b25808aa55087fb5555 P 9fbd96f2c37b4a6a89fdca1756ab8a98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:27:14.485666 18969 ts_tablet_manager.cc:1905] T 41945de770d54b25808aa55087fb5555 P 638feafd83954241a681cb7e2ab8ec64: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:27:14.488212 18969 ts_tablet_manager.cc:1918] T 41945de770d54b25808aa55087fb5555 P 638feafd83954241a681cb7e2ab8ec64: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:27:14.488309 18969 log.cc:1196] T 41945de770d54b25808aa55087fb5555 P 638feafd83954241a681cb7e2ab8ec64: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-0/wal/wals/41945de770d54b25808aa55087fb5555
I20260808 22:27:14.488397 18968 ts_tablet_manager.cc:1918] T 41945de770d54b25808aa55087fb5555 P 9fbd96f2c37b4a6a89fdca1756ab8a98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260808 22:27:14.488478 18968 log.cc:1196] T 41945de770d54b25808aa55087fb5555 P 9fbd96f2c37b4a6a89fdca1756ab8a98: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786227991988852-5167-0/minicluster-data/ts-2/wal/wals/41945de770d54b25808aa55087fb5555
I20260808 22:27:14.488596 18969 ts_tablet_manager.cc:1939] T 41945de770d54b25808aa55087fb5555 P 638feafd83954241a681cb7e2ab8ec64: Deleting consensus metadata
I20260808 22:27:14.488755 18968 ts_tablet_manager.cc:1939] T 41945de770d54b25808aa55087fb5555 P 9fbd96f2c37b4a6a89fdca1756ab8a98: Deleting consensus metadata
I20260808 22:27:14.489434 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 18217
I20260808 22:27:14.497834 18073 catalog_manager.cc:4841] TS 9fbd96f2c37b4a6a89fdca1756ab8a98 (127.5.11.195:37603): tablet 41945de770d54b25808aa55087fb5555 (table test-table [id=0c9edcddc9fd4f8e9c9b57da9aca466d]) successfully deleted
I20260808 22:27:14.498046 18073 catalog_manager.cc:4841] TS 638feafd83954241a681cb7e2ab8ec64 (127.5.11.193:43025): tablet 41945de770d54b25808aa55087fb5555 (table test-table [id=0c9edcddc9fd4f8e9c9b57da9aca466d]) successfully deleted
I20260808 22:27:14.498633 18073 catalog_manager.cc:4841] TS 7850c97cdd9d4c37b45acbccf1003368 (127.5.11.194:45435): tablet 41945de770d54b25808aa55087fb5555 (table test-table [id=0c9edcddc9fd4f8e9c9b57da9aca466d]) successfully deleted
I20260808 22:27:14.505766 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 18442
I20260808 22:27:14.519811 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 18676
I20260808 22:27:14.531808 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 18046
2026-08-08T22:27:14Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3627 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:27:14 dist-test-slave-7f01 krb5kdc[19089](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:27:14 dist-test-slave-7f01 krb5kdc[19089](info): set up 2 sockets
Aug 08 22:27:14 dist-test-slave-7f01 krb5kdc[19089](info): commencing operation
krb5kdc: starting...
W20260808 22:27:15.781802 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:27:15 dist-test-slave-7f01 krb5kdc[19089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228035, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:27:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:27:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.11.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
I20260808 22:27:15.906653 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:37027
--webserver_interface=127.5.11.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:46755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:37027
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:27:16.020655 19104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:16.020901 19104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:16.020941 19104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:16.024940 19104 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:27:16.024999 19104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:16.025019 19104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:16.025035 19104 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:27:16.025050 19104 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:27:16.030134 19104 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:46755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:37027
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:37027
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:16.031097 19104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:16.032192 19104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:16.040465 19117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:16.040477 19119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:16.040504 19115 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:16.040856 19104 server_base.cc:1034] running on GCE node
I20260808 22:27:16.041420 19104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:16.042572 19104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:16.043802 19104 hybrid_clock.cc:648] HybridClock initialized: now 1786228036043779 us; error 30 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-08T22:27:16Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1518 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:27:16 dist-test-slave-7f01 krb5kdc[19127](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:27:16 dist-test-slave-7f01 krb5kdc[19127](info): set up 2 sockets
Aug 08 22:27:16 dist-test-slave-7f01 krb5kdc[19127](info): commencing operation
krb5kdc: starting...
W20260808 22:27:17.302974 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.226s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:27:17 dist-test-slave-7f01 krb5kdc[19127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:27:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:27:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.11.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
I20260808 22:27:17.430814 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:43839
--webserver_interface=127.5.11.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:38709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:43839
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:27:17.541110 19152 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:17.541373 19152 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:17.541443 19152 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:17.545315 19152 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:27:17.545418 19152 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:17.545456 19152 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:17.545519 19152 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:27:17.545557 19152 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:27:17.550678 19152 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:38709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:43839
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:43839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19152
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:17.551659 19152 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:17.552662 19152 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:17.560976 19163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:17.560964 19159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:17.560984 19158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:17.561367 19152 server_base.cc:1034] running on GCE node
I20260808 22:27:17.561911 19152 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:17.562906 19152 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:17.564100 19152 hybrid_clock.cc:648] HybridClock initialized: now 1786228037564088 us; error 29 us; skew 500 ppm
Aug 08 22:27:17 dist-test-slave-7f01 krb5kdc[19127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228037, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:27:17.570407 19152 init.cc:377] Logged in from keytab as kudu/127.5.11.254@KRBTEST.COM (short username kudu)
I20260808 22:27:17.571282 19152 webserver.cc:460] Webserver started at http://127.5.11.254:46559/ using document root <none> and password file <none>
I20260808 22:27:17.571868 19152 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:17.571936 19152 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:17.572109 19152 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:27:17.574083 19152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "8f61e7b96b1b44b4861673aad59b917e"
format_stamp: "Formatted at 2026-08-08 22:27:17 on dist-test-slave-7f01"
server_key: "be89fe3ff75b3b2b1cb4c3265d82d190"
server_key_iv: "0f73adbd22255dc4d5715b6adea5f608"
server_key_version: "encryptionkey@0"
I20260808 22:27:17.574592 19152 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "8f61e7b96b1b44b4861673aad59b917e"
format_stamp: "Formatted at 2026-08-08 22:27:17 on dist-test-slave-7f01"
server_key: "be89fe3ff75b3b2b1cb4c3265d82d190"
server_key_iv: "0f73adbd22255dc4d5715b6adea5f608"
server_key_version: "encryptionkey@0"
I20260808 22:27:17.578188 19152 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 22:27:17.580473 19174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:17.581653 19152 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:27:17.581794 19152 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "8f61e7b96b1b44b4861673aad59b917e"
format_stamp: "Formatted at 2026-08-08 22:27:17 on dist-test-slave-7f01"
server_key: "be89fe3ff75b3b2b1cb4c3265d82d190"
server_key_iv: "0f73adbd22255dc4d5715b6adea5f608"
server_key_version: "encryptionkey@0"
I20260808 22:27:17.581895 19152 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:17.593868 19152 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:17.596576 19152 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:17.596771 19152 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:17.604076 19152 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:43839
I20260808 22:27:17.604090 19257 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:43839 every 8 connection(s)
I20260808 22:27:17.605002 19152 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:27:17.606838 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 19152
I20260808 22:27:17.606982 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:27:17.607299 5167 external_mini_cluster.cc:1428] Setting key 94a3d415dd711101369ee90c77a8fbba
I20260808 22:27:17.608256 19259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:27:17.614838 19259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e: Bootstrap starting.
Aug 08 22:27:17 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228037, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:27:17.617434 19259 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e: Neither blocks nor log segments found. Creating new log.
I20260808 22:27:17.618166 19259 log.cc:824] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e: Log is configured to *not* fsync() on all Append() calls
I20260808 22:27:17.620132 19259 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e: No bootstrap required, opened a new log
I20260808 22:27:17.621768 19262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:17.608677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43036 (local address 127.5.11.254:43839)
0808 22:27:17.609073 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:17.609095 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:17.609143 (+ 48us) server_negotiation.cc:407] Connection header received
0808 22:27:17.609757 (+ 614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:17.609772 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:17.610072 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:17.610334 (+ 262us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:17.611269 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:17.612321 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:17.613500 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:17.613945 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:17.614056 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:17.616914 (+ 2858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:17.616919 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:17.616930 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:17.616957 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:17.618939 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:17.619408 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:17.619414 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:17.619420 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:17.619543 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:17.619873 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:17.619877 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:17.619878 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:17.620311 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:17.620517 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:17.620819 (+ 302us) server_negotiation.cc:299] Negotiation successful
0808 22:27:17.621109 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":140,"threads_started":1}
I20260808 22:27:17.624045 19259 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f61e7b96b1b44b4861673aad59b917e" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 43839 } }
I20260808 22:27:17.624321 19259 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:27:17.624418 19259 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f61e7b96b1b44b4861673aad59b917e, State: Initialized, Role: FOLLOWER
I20260808 22:27:17.624814 19259 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f61e7b96b1b44b4861673aad59b917e" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 43839 } }
I20260808 22:27:17.624970 19259 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:27:17.625066 19259 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:27:17.625168 19259 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:27:17.626256 19259 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f61e7b96b1b44b4861673aad59b917e" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 43839 } }
I20260808 22:27:17.626559 19259 leader_election.cc:302] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f61e7b96b1b44b4861673aad59b917e; no voters:
I20260808 22:27:17.626840 19259 leader_election.cc:288] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:27:17.626922 19266 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:27:17.627179 19266 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e [term 1 LEADER]: Becoming Leader. State: Replica: 8f61e7b96b1b44b4861673aad59b917e, State: Running, Role: LEADER
I20260808 22:27:17.627492 19266 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f61e7b96b1b44b4861673aad59b917e" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 43839 } }
I20260808 22:27:17.628103 19259 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:27:17.629256 19269 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f61e7b96b1b44b4861673aad59b917e. Latest consensus state: current_term: 1 leader_uuid: "8f61e7b96b1b44b4861673aad59b917e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f61e7b96b1b44b4861673aad59b917e" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 43839 } } }
I20260808 22:27:17.629434 19269 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e [sys.catalog]: This master's current role is: LEADER
I20260808 22:27:17.629258 19268 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f61e7b96b1b44b4861673aad59b917e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f61e7b96b1b44b4861673aad59b917e" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 43839 } } }
I20260808 22:27:17.629684 19268 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e [sys.catalog]: This master's current role is: LEADER
I20260808 22:27:17.629849 19283 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:27:17.632632 19283 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:27:17.637956 19283 catalog_manager.cc:1347] Generated new cluster ID: f6d142f8b2eb4f7694bd529d4ec01591
I20260808 22:27:17.638259 19283 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:27:17.657266 19283 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:27:17.658216 19283 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:27:17.664175 19283 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e: Generated new TSK 0
I20260808 22:27:17.664819 19283 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.11.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab.
Entry for principal kudu/127.5.11.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab.
I20260808 22:27:17.703047 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.193:0
--local_ip_for_outbound_sockets=127.5.11.193
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:43839
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:38709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:27:17.815874 19304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:17.816118 19304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:17.816159 19304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:17.820302 19304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:17.820351 19304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:17.820446 19304 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.193
I20260808 22:27:17.825373 19304 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:38709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:43839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.193
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:17.826306 19304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:17.827312 19304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:17.837050 19310 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:17.837128 19304 server_base.cc:1034] running on GCE node
W20260808 22:27:17.837033 19311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:17.837045 19313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:17.837654 19304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:17.838205 19304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:17.839368 19304 hybrid_clock.cc:648] HybridClock initialized: now 1786228037839346 us; error 32 us; skew 500 ppm
Aug 08 22:27:17 dist-test-slave-7f01 krb5kdc[19127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228037, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:27:17.842218 19304 init.cc:377] Logged in from keytab as kudu/127.5.11.193@KRBTEST.COM (short username kudu)
I20260808 22:27:17.843040 19304 webserver.cc:460] Webserver started at http://127.5.11.193:46561/ using document root <none> and password file <none>
I20260808 22:27:17.843590 19304 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:17.843681 19304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:17.843938 19304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:27:17.845877 19304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-0/data/instance:
uuid: "c710317a4d39471c84b6e1d13deeb9ed"
format_stamp: "Formatted at 2026-08-08 22:27:17 on dist-test-slave-7f01"
server_key: "2841d0253ce761ab8a67c0caa6351a79"
server_key_iv: "c1f44a2623e7734b1e1fb6eca9877c17"
server_key_version: "encryptionkey@0"
I20260808 22:27:17.846446 19304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance:
uuid: "c710317a4d39471c84b6e1d13deeb9ed"
format_stamp: "Formatted at 2026-08-08 22:27:17 on dist-test-slave-7f01"
server_key: "2841d0253ce761ab8a67c0caa6351a79"
server_key_iv: "c1f44a2623e7734b1e1fb6eca9877c17"
server_key_version: "encryptionkey@0"
I20260808 22:27:17.850126 19304 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 22:27:17.852509 19323 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:17.853624 19304 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:27:17.853765 19304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-0/wal
uuid: "c710317a4d39471c84b6e1d13deeb9ed"
format_stamp: "Formatted at 2026-08-08 22:27:17 on dist-test-slave-7f01"
server_key: "2841d0253ce761ab8a67c0caa6351a79"
server_key_iv: "c1f44a2623e7734b1e1fb6eca9877c17"
server_key_version: "encryptionkey@0"
I20260808 22:27:17.853874 19304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:17.865872 19304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:17.869113 19304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:17.869315 19304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:17.869882 19304 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:27:17.870690 19304 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:27:17.870764 19304 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:17.870828 19304 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:27:17.870872 19304 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:17.881270 19304 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.193:35239
I20260808 22:27:17.881299 19510 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.193:35239 every 8 connection(s)
I20260808 22:27:17.882578 19304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
I20260808 22:27:17.889459 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 19304
I20260808 22:27:17.889628 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance
I20260808 22:27:17.889961 5167 external_mini_cluster.cc:1428] Setting key 026bfa0f16cd4b81a04deae08c1f3053
Aug 08 22:27:17 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228037, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:27:17.897991 19262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:17.885301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:36709 (local address 127.5.11.254:43839)
0808 22:27:17.885511 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:17.885516 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:17.886437 (+ 921us) server_negotiation.cc:407] Connection header received
0808 22:27:17.887463 (+ 1026us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:17.887466 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:17.887517 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:17.887594 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:17.888793 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:17.889455 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:17.890872 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:17.891127 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:17.891241 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:17.894009 (+ 2768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:17.894014 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:17.894018 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:17.894054 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:17.895773 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:17.896278 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:17.896284 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:17.896289 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:17.896401 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:17.896754 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:17.896760 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:17.896764 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:17.897064 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:17.897181 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:17.897653 (+ 472us) server_negotiation.cc:299] Negotiation successful
0808 22:27:17.897795 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 22:27:17.898684 19515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:17.885389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.193:36709)
0808 22:27:17.886272 (+ 883us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:17.886334 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:17.887261 (+ 927us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:17.887727 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:17.887778 (+ 51us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:17.888170 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:17.888655 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:17.888670 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:17.889617 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:17.889621 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:17.890738 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:17.890748 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:17.891298 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:17.891302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:17.891475 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:17.892132 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:17.892148 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:17.893837 (+ 1689us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:17.895897 (+ 2060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:17.895901 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:17.895913 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:17.896169 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:17.896502 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:17.896504 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:17.896505 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:17.896655 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:17.897175 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:17.897180 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:17.897400 (+ 220us) client_negotiation.cc:769] Sending connection context
0808 22:27:17.897559 (+ 159us) client_negotiation.cc:240] Negotiation successful
0808 22:27:17.897799 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":155,"threads_started":1}
I20260808 22:27:17.899951 19511 heartbeater.cc:347] Connected to a master server at 127.5.11.254:43839
I20260808 22:27:17.900219 19511 heartbeater.cc:464] Registering TS with master...
I20260808 22:27:17.900836 19511 heartbeater.cc:511] Master 127.5.11.254:43839 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.5.11.194@KRBTEST.COM; defaulting to no policy
I20260808 22:27:17.902516 19199 ts_manager.cc:194] Registered new tserver with Master: c710317a4d39471c84b6e1d13deeb9ed (127.5.11.193:35239)
I20260808 22:27:17.903816 19199 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.193@KRBTEST.COM'} at 127.5.11.193:36709
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab.
Entry for principal kudu/127.5.11.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab.
I20260808 22:27:17.918689 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.194:0
--local_ip_for_outbound_sockets=127.5.11.194
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:43839
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:38709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:27:18.032173 19520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:18.032418 19520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:18.032457 19520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:18.036675 19520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:18.036731 19520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:18.036787 19520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.194
I20260808 22:27:18.041872 19520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:38709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:43839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.194
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:18.042758 19520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:18.043717 19520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:18.053015 19533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:18.053051 19527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:18.053062 19529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:18.053369 19520 server_base.cc:1034] running on GCE node
I20260808 22:27:18.053750 19520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:18.054379 19520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:18.055540 19520 hybrid_clock.cc:648] HybridClock initialized: now 1786228038055509 us; error 41 us; skew 500 ppm
Aug 08 22:27:18 dist-test-slave-7f01 krb5kdc[19127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228038, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:27:18.058598 19520 init.cc:377] Logged in from keytab as kudu/127.5.11.194@KRBTEST.COM (short username kudu)
I20260808 22:27:18.059423 19520 webserver.cc:460] Webserver started at http://127.5.11.194:45161/ using document root <none> and password file <none>
I20260808 22:27:18.060071 19520 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:18.060165 19520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:18.060396 19520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:27:18.062390 19520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-1/data/instance:
uuid: "ed214bafe47e45d7b4582daee820e6a9"
format_stamp: "Formatted at 2026-08-08 22:27:18 on dist-test-slave-7f01"
server_key: "aee183e9e0c565cf838a4aa5d4855a74"
server_key_iv: "63cbeddf5ac209a2c308f6f9137aa5d0"
server_key_version: "encryptionkey@0"
I20260808 22:27:18.062955 19520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance:
uuid: "ed214bafe47e45d7b4582daee820e6a9"
format_stamp: "Formatted at 2026-08-08 22:27:18 on dist-test-slave-7f01"
server_key: "aee183e9e0c565cf838a4aa5d4855a74"
server_key_iv: "63cbeddf5ac209a2c308f6f9137aa5d0"
server_key_version: "encryptionkey@0"
I20260808 22:27:18.066715 19520 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260808 22:27:18.069214 19546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:18.070410 19520 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:27:18.070551 19520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-1/wal
uuid: "ed214bafe47e45d7b4582daee820e6a9"
format_stamp: "Formatted at 2026-08-08 22:27:18 on dist-test-slave-7f01"
server_key: "aee183e9e0c565cf838a4aa5d4855a74"
server_key_iv: "63cbeddf5ac209a2c308f6f9137aa5d0"
server_key_version: "encryptionkey@0"
I20260808 22:27:18.070663 19520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:18.088685 19520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:18.091920 19520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:18.092123 19520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:18.092689 19520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:27:18.093495 19520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:27:18.093569 19520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:18.093640 19520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:27:18.093694 19520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:18.102902 19520 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.194:39107
I20260808 22:27:18.102931 19735 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.194:39107 every 8 connection(s)
I20260808 22:27:18.103811 19520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
I20260808 22:27:18.104779 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 19520
I20260808 22:27:18.104933 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance
I20260808 22:27:18.105258 5167 external_mini_cluster.cc:1428] Setting key 84cba9c3caef4fe5a9a0608ffeaf705e
Aug 08 22:27:18 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228038, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.11.195@KRBTEST.COM; defaulting to no policy
I20260808 22:27:18.118077 19262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:18.105904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:53789 (local address 127.5.11.254:43839)
0808 22:27:18.106067 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:18.106072 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:18.107225 (+ 1153us) server_negotiation.cc:407] Connection header received
0808 22:27:18.108181 (+ 956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:18.108187 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:18.108252 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:18.108348 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:18.109464 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:18.109987 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:18.111122 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:18.111348 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:18.111462 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:18.114346 (+ 2884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:18.114350 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:18.114352 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:18.114375 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:18.115919 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:18.116536 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:18.116539 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:18.116540 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:18.116619 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:18.117066 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:18.117071 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:18.117073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:18.117311 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:18.117393 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:18.117851 (+ 458us) server_negotiation.cc:299] Negotiation successful
0808 22:27:18.117948 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.195@KRBTEST.COM" created.
I20260808 22:27:18.119019 19738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:18.106313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.194:53789)
0808 22:27:18.107058 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:18.107111 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:18.107981 (+ 870us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:18.108511 (+ 530us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:18.108517 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:18.108855 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:18.109315 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:18.109327 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:18.110123 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:18.110126 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:18.110990 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:18.111001 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:18.111509 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:18.111514 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:18.111685 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:18.112330 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:18.112347 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:18.114169 (+ 1822us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:18.116035 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:18.116043 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:18.116059 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:18.116436 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:18.116707 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:18.116710 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:18.116712 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:18.116947 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:18.117410 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:18.117416 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:18.117638 (+ 222us) client_negotiation.cc:769] Sending connection context
0808 22:27:18.117795 (+ 157us) client_negotiation.cc:240] Negotiation successful
0808 22:27:18.118055 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":143,"threads_started":1}
I20260808 22:27:18.120282 19736 heartbeater.cc:347] Connected to a master server at 127.5.11.254:43839
I20260808 22:27:18.120527 19736 heartbeater.cc:464] Registering TS with master...
I20260808 22:27:18.121142 19736 heartbeater.cc:511] Master 127.5.11.254:43839 requested a full tablet report, sending...
I20260808 22:27:18.122357 19199 ts_manager.cc:194] Registered new tserver with Master: ed214bafe47e45d7b4582daee820e6a9 (127.5.11.194:39107)
I20260808 22:27:18.122939 19199 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.194@KRBTEST.COM'} at 127.5.11.194:53789
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab.
Entry for principal kudu/127.5.11.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab.
I20260808 22:27:18.134469 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.195:0
--local_ip_for_outbound_sockets=127.5.11.195
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:43839
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:38709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:27:18.245163 19743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:18.245432 19743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:18.245507 19743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:18.249635 19743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:18.249711 19743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:18.249794 19743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.195
I20260808 22:27:18.254830 19743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:38709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:43839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19743
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.195
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:18.255848 19743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:18.256795 19743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:18.266280 19751 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:18.266283 19748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:18.266281 19749 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:18.266610 19743 server_base.cc:1034] running on GCE node
I20260808 22:27:18.267050 19743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:18.267671 19743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:18.268839 19743 hybrid_clock.cc:648] HybridClock initialized: now 1786228038268797 us; error 49 us; skew 500 ppm
Aug 08 22:27:18 dist-test-slave-7f01 krb5kdc[19127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228038, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:27:18.271607 19743 init.cc:377] Logged in from keytab as kudu/127.5.11.195@KRBTEST.COM (short username kudu)
I20260808 22:27:18.272493 19743 webserver.cc:460] Webserver started at http://127.5.11.195:45197/ using document root <none> and password file <none>
I20260808 22:27:18.273051 19743 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:18.273144 19743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:18.273420 19743 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:27:18.275566 19743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-2/data/instance:
uuid: "a6be3d39bfc847d399c62c4ea378fd17"
format_stamp: "Formatted at 2026-08-08 22:27:18 on dist-test-slave-7f01"
server_key: "5eddac08fc0b8ebff4b16c4a364083e3"
server_key_iv: "4149427c577c62eb2608bc1a13ae1de4"
server_key_version: "encryptionkey@0"
I20260808 22:27:18.276233 19743 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance:
uuid: "a6be3d39bfc847d399c62c4ea378fd17"
format_stamp: "Formatted at 2026-08-08 22:27:18 on dist-test-slave-7f01"
server_key: "5eddac08fc0b8ebff4b16c4a364083e3"
server_key_iv: "4149427c577c62eb2608bc1a13ae1de4"
server_key_version: "encryptionkey@0"
I20260808 22:27:18.279976 19743 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:27:18.282289 19762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:18.283447 19743 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:27:18.283586 19743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-2/wal
uuid: "a6be3d39bfc847d399c62c4ea378fd17"
format_stamp: "Formatted at 2026-08-08 22:27:18 on dist-test-slave-7f01"
server_key: "5eddac08fc0b8ebff4b16c4a364083e3"
server_key_iv: "4149427c577c62eb2608bc1a13ae1de4"
server_key_version: "encryptionkey@0"
I20260808 22:27:18.283694 19743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:18.321841 19743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:18.325218 19743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:18.325434 19743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:18.326047 19743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:27:18.326900 19743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:27:18.326974 19743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:18.327047 19743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:27:18.327100 19743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:18.336345 19743 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.195:46677
I20260808 22:27:18.336390 19960 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.195:46677 every 8 connection(s)
I20260808 22:27:18.337225 19743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
I20260808 22:27:18.340632 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 19743
I20260808 22:27:18.340773 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance
I20260808 22:27:18.341070 5167 external_mini_cluster.cc:1428] Setting key 74f78622d621a495de9b46601c6aa9c9
Aug 08 22:27:18 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228038, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:27:18.350924 19262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:18.338963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:53227 (local address 127.5.11.254:43839)
0808 22:27:18.339099 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:18.339103 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:18.340143 (+ 1040us) server_negotiation.cc:407] Connection header received
0808 22:27:18.341321 (+ 1178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:18.341324 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:18.341375 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:18.341496 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:18.342931 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:18.343585 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:18.344733 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:18.344945 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:18.345027 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:18.347294 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:18.347298 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:18.347300 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:18.347323 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:18.348776 (+ 1453us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:18.349257 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:18.349261 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:18.349262 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:18.349343 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:18.349690 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:18.349694 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:18.349696 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:18.349923 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:18.350033 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:18.350642 (+ 609us) server_negotiation.cc:299] Negotiation successful
0808 22:27:18.350761 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 22:27:18.351722 19966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:18.339296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.195:53227)
0808 22:27:18.339994 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:18.340036 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:18.341085 (+ 1049us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:18.341690 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:18.341699 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:18.342193 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:18.342791 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:18.342805 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:18.343704 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:18.343707 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:18.344619 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:18.344629 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:18.345073 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:18.345076 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:18.345211 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:18.345715 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:18.345731 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:18.347143 (+ 1412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:18.348873 (+ 1730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:18.348878 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:18.348888 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:18.349133 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:18.349437 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:18.349441 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:18.349444 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:18.349581 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:18.350039 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:18.350047 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:18.350391 (+ 344us) client_negotiation.cc:769] Sending connection context
0808 22:27:18.350583 (+ 192us) client_negotiation.cc:240] Negotiation successful
0808 22:27:18.350879 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":111,"threads_started":1}
I20260808 22:27:18.352833 19961 heartbeater.cc:347] Connected to a master server at 127.5.11.254:43839
I20260808 22:27:18.353047 19961 heartbeater.cc:464] Registering TS with master...
I20260808 22:27:18.353531 19961 heartbeater.cc:511] Master 127.5.11.254:43839 requested a full tablet report, sending...
I20260808 22:27:18.354532 19199 ts_manager.cc:194] Registered new tserver with Master: a6be3d39bfc847d399c62c4ea378fd17 (127.5.11.195:46677)
I20260808 22:27:18.355002 5167 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:27:18.355116 19199 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.195@KRBTEST.COM'} at 127.5.11.195:53227
WARNING: no policy specified for oryx/127.5.11.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.11.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.11.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.193.keytab.
Entry for principal oryx/127.5.11.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.193.keytab.
WARNING: no policy specified for oryx/127.5.11.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.11.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.11.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.194.keytab.
Entry for principal oryx/127.5.11.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.194.keytab.
WARNING: no policy specified for oryx/127.5.11.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.5.11.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.5.11.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.195.keytab.
Entry for principal oryx/127.5.11.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.195.keytab.
I20260808 22:27:18.428433 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 19304
I20260808 22:27:18.445436 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 19520
I20260808 22:27:18.460270 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 19743
I20260808 22:27:18.469532 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 19152
I20260808 22:27:18.481762 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:43839
--webserver_interface=127.5.11.254
--webserver_port=46559
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:38709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:43839
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:27:18.592979 19974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:18.593267 19974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:18.593357 19974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:18.597299 19974 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:27:18.597373 19974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:18.597411 19974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:18.597467 19974 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:27:18.597502 19974 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:27:18.602596 19974 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:38709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:43839
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:43839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=46559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:18.603590 19974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:18.604642 19974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:18.613157 19987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:18.613166 19990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:18.613220 19974 server_base.cc:1034] running on GCE node
W20260808 22:27:18.613168 19988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:18.613986 19974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:18.614977 19974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:18.616137 19974 hybrid_clock.cc:648] HybridClock initialized: now 1786228038616103 us; error 44 us; skew 500 ppm
Aug 08 22:27:18 dist-test-slave-7f01 krb5kdc[19127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228038, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:27:18.619107 19974 init.cc:377] Logged in from keytab as kudu/127.5.11.254@KRBTEST.COM (short username kudu)
I20260808 22:27:18.619993 19974 webserver.cc:460] Webserver started at http://127.5.11.254:46559/ using document root <none> and password file <none>
I20260808 22:27:18.620508 19974 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:18.620581 19974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:18.624241 19974 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 22:27:18.626215 19997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:18.627447 19974 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:27:18.627601 19974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "8f61e7b96b1b44b4861673aad59b917e"
format_stamp: "Formatted at 2026-08-08 22:27:17 on dist-test-slave-7f01"
server_key: "be89fe3ff75b3b2b1cb4c3265d82d190"
server_key_iv: "0f73adbd22255dc4d5715b6adea5f608"
server_key_version: "encryptionkey@0"
I20260808 22:27:18.628232 19974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:18.638788 19974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:18.641475 19974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:18.641678 19974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:18.648552 19974 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:43839
I20260808 22:27:18.648587 20076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:43839 every 8 connection(s)
I20260808 22:27:18.649582 19974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:27:18.653682 20077 sys_catalog.cc:263] Verifying existing consensus state
I20260808 22:27:18.655283 20077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e: Bootstrap starting.
I20260808 22:27:18.658578 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 19974
I20260808 22:27:18.659080 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.193:35239
--local_ip_for_outbound_sockets=127.5.11.193
--tserver_master_addrs=127.5.11.254:43839
--webserver_port=46561
--webserver_interface=127.5.11.193
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:38709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.11.193
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:27:18.673110 20077 log.cc:824] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e: Log is configured to *not* fsync() on all Append() calls
I20260808 22:27:18.679814 20077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 22:27:18.680249 20077 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e: Bootstrap complete.
I20260808 22:27:18.684814 20077 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f61e7b96b1b44b4861673aad59b917e" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 43839 } }
I20260808 22:27:18.685282 20077 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f61e7b96b1b44b4861673aad59b917e, State: Initialized, Role: FOLLOWER
I20260808 22:27:18.685845 20077 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f61e7b96b1b44b4861673aad59b917e" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 43839 } }
I20260808 22:27:18.685997 20077 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:27:18.686100 20077 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:27:18.686200 20077 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e [term 1 FOLLOWER]: Advancing to term 2
I20260808 22:27:18.688213 20077 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f61e7b96b1b44b4861673aad59b917e" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 43839 } }
I20260808 22:27:18.688557 20077 leader_election.cc:302] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f61e7b96b1b44b4861673aad59b917e; no voters:
I20260808 22:27:18.688908 20077 leader_election.cc:288] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 22:27:18.688988 20083 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e [term 2 FOLLOWER]: Leader election won for term 2
I20260808 22:27:18.689215 20083 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e [term 2 LEADER]: Becoming Leader. State: Replica: 8f61e7b96b1b44b4861673aad59b917e, State: Running, Role: LEADER
I20260808 22:27:18.689519 20083 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f61e7b96b1b44b4861673aad59b917e" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 43839 } }
I20260808 22:27:18.690192 20077 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:27:18.690549 20087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8f61e7b96b1b44b4861673aad59b917e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f61e7b96b1b44b4861673aad59b917e" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 43839 } } }
I20260808 22:27:18.690691 20087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e [sys.catalog]: This master's current role is: LEADER
I20260808 22:27:18.690915 20084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f61e7b96b1b44b4861673aad59b917e. Latest consensus state: current_term: 2 leader_uuid: "8f61e7b96b1b44b4861673aad59b917e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f61e7b96b1b44b4861673aad59b917e" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 43839 } } }
I20260808 22:27:18.691039 20084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e [sys.catalog]: This master's current role is: LEADER
I20260808 22:27:18.691346 20091 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:27:18.694670 20091 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:27:18.695406 20091 catalog_manager.cc:1259] Loaded cluster ID: f6d142f8b2eb4f7694bd529d4ec01591
I20260808 22:27:18.695467 20091 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:27:18.697889 20091 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:27:18.698565 20091 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8f61e7b96b1b44b4861673aad59b917e: Loaded TSK: 0
I20260808 22:27:18.699805 20091 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 22:27:18.774755 20079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:18.775027 20079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:18.775112 20079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:18.779172 20079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:18.779258 20079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:18.779351 20079 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.193
I20260808 22:27:18.784462 20079 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:38709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.193.keytab
--principal=oryx/127.5.11.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.193:35239
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.11.193
--webserver_port=46561
--tserver_master_addrs=127.5.11.254:43839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20079
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.193
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:18.785454 20079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:18.786479 20079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:18.797043 20114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:18.797065 20117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:18.797152 20119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:18.797413 20079 server_base.cc:1034] running on GCE node
I20260808 22:27:18.797880 20079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:18.798535 20079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:18.799729 20079 hybrid_clock.cc:648] HybridClock initialized: now 1786228038799705 us; error 43 us; skew 500 ppm
Aug 08 22:27:18 dist-test-slave-7f01 krb5kdc[19127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228038, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:27:18.802743 20079 init.cc:377] Logged in from keytab as oryx/127.5.11.193@KRBTEST.COM (short username oryx)
I20260808 22:27:18.803651 20079 webserver.cc:460] Webserver started at http://127.5.11.193:46561/ using document root <none> and password file <none>
I20260808 22:27:18.804244 20079 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:18.804359 20079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:18.808387 20079 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:27:18.810616 20130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:18.811909 20079 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:27:18.812042 20079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-0/wal
uuid: "c710317a4d39471c84b6e1d13deeb9ed"
format_stamp: "Formatted at 2026-08-08 22:27:17 on dist-test-slave-7f01"
server_key: "2841d0253ce761ab8a67c0caa6351a79"
server_key_iv: "c1f44a2623e7734b1e1fb6eca9877c17"
server_key_version: "encryptionkey@0"
I20260808 22:27:18.812579 20079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:18.846530 20079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:18.849397 20079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:18.849603 20079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:18.850194 20079 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:27:18.851019 20079 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:27:18.851096 20079 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:18.851171 20079 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:27:18.851217 20079 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:18.860719 20079 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.193:35239
I20260808 22:27:18.860746 20314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.193:35239 every 8 connection(s)
I20260808 22:27:18.861692 20079 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
I20260808 22:27:18.866761 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 20079
I20260808 22:27:18.867233 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.194:39107
--local_ip_for_outbound_sockets=127.5.11.194
--tserver_master_addrs=127.5.11.254:43839
--webserver_port=45161
--webserver_interface=127.5.11.194
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:38709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.11.194
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 22:27:18 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:18 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
W20260808 22:27:18.875144 20319 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:18.863833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:42675 (local address 127.5.11.254:43839)
0808 22:27:18.864395 (+ 562us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:18.864418 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:18.864828 (+ 410us) server_negotiation.cc:407] Connection header received
0808 22:27:18.865929 (+ 1101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:18.865940 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:18.866231 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:18.866574 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:18.867904 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:18.868875 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:18.870028 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:18.870517 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:18.870703 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:18.874478 (+ 3775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.193:42675: BlockingRecv error: recv got EOF from 127.5.11.193:42675 (error 108)
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":127,"threads_started":1}
I20260808 22:27:18.875293 20321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:18.863966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.193:42675)
0808 22:27:18.864672 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:18.864713 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:18.865521 (+ 808us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:18.866662 (+ 1141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:18.866670 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:18.867145 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:18.867684 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:18.867700 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:18.869031 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:18.869040 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:18.869897 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:18.869907 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:18.870693 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:18.870698 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:18.870879 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:18.871478 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:18.871494 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:18.874292 (+ 2798us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":132,"threads_started":1}
W20260808 22:27:18.875725 20316 heartbeater.cc:644] Failed to heartbeat to 127.5.11.254:43839 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.11.254:43839: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Aug 08 22:27:18 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:18 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:18.882019 20321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:18.876257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.193:59951)
0808 22:27:18.876446 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:18.876464 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:18.876547 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:18.876793 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:18.876798 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:18.876995 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:18.877139 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:18.877149 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:18.877833 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:18.877840 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:18.878838 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:18.878851 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:18.879275 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:18.879281 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:18.879419 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:18.879950 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:18.879971 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:18.881876 (+ 1905us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108}
W20260808 22:27:18.882103 20319 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:18.876346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:59951 (local address 127.5.11.254:43839)
0808 22:27:18.876493 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:18.876497 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:18.876515 (+ 18us) server_negotiation.cc:407] Connection header received
0808 22:27:18.876637 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:18.876639 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:18.876675 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:18.876741 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:18.877252 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:18.877729 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:18.878968 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:18.879163 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:18.879234 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:18.881991 (+ 2757us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.193:59951: BlockingRecv error: recv got EOF from 127.5.11.193:59951 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 08 22:27:18 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:18 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:18.887387 20321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:18.882673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.193:57645)
0808 22:27:18.882762 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:18.882777 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:18.882849 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:18.883071 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:18.883074 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:18.883227 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:18.883291 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:18.883296 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:18.884046 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:18.884051 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:18.884544 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:18.884551 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:18.884999 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:18.885003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:18.885092 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:18.885522 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:18.885543 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:18.887258 (+ 1715us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":27}
W20260808 22:27:18.887554 20316 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 22:27:18.887473 20319 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:18.882758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:57645 (local address 127.5.11.254:43839)
0808 22:27:18.882864 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:18.882870 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:18.882888 (+ 18us) server_negotiation.cc:407] Connection header received
0808 22:27:18.882924 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:18.882927 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:18.882968 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:18.883057 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:18.883404 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:18.883939 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:18.884682 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:18.884889 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:18.884995 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:18.887366 (+ 2371us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.193:57645: BlockingRecv error: recv got EOF from 127.5.11.193:57645 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260808 22:27:18.983628 20327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:18.983935 20327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:18.984019 20327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:18.987973 20327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:18.988059 20327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:18.988143 20327 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.194
I20260808 22:27:18.993681 20327 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:38709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.194.keytab
--principal=oryx/127.5.11.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.194:39107
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.11.194
--webserver_port=45161
--tserver_master_addrs=127.5.11.254:43839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.194
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:18.994673 20327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:18.995671 20327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:19.005576 20334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:19.005589 20338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:19.005576 20335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:19.005818 20327 server_base.cc:1034] running on GCE node
I20260808 22:27:19.006268 20327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:19.006943 20327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:19.008101 20327 hybrid_clock.cc:648] HybridClock initialized: now 1786228039008087 us; error 24 us; skew 500 ppm
Aug 08 22:27:19 dist-test-slave-7f01 krb5kdc[19127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228039, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:27:19.010788 20327 init.cc:377] Logged in from keytab as oryx/127.5.11.194@KRBTEST.COM (short username oryx)
I20260808 22:27:19.011634 20327 webserver.cc:460] Webserver started at http://127.5.11.194:45161/ using document root <none> and password file <none>
I20260808 22:27:19.012202 20327 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:19.012279 20327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:19.016590 20327 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:27:19.019201 20349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:19.020411 20327 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:27:19.020553 20327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-1/wal
uuid: "ed214bafe47e45d7b4582daee820e6a9"
format_stamp: "Formatted at 2026-08-08 22:27:18 on dist-test-slave-7f01"
server_key: "aee183e9e0c565cf838a4aa5d4855a74"
server_key_iv: "63cbeddf5ac209a2c308f6f9137aa5d0"
server_key_version: "encryptionkey@0"
I20260808 22:27:19.021083 20327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:19.037497 20327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:19.040620 20327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:19.040817 20327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:19.041378 20327 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:27:19.042179 20327 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:27:19.042253 20327 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:19.042322 20327 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:27:19.042375 20327 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:19.051801 20327 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.194:39107
I20260808 22:27:19.051855 20526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.194:39107 every 8 connection(s)
I20260808 22:27:19.052747 20327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
I20260808 22:27:19.054399 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 20327
I20260808 22:27:19.054879 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.195:46677
--local_ip_for_outbound_sockets=127.5.11.195
--tserver_master_addrs=127.5.11.254:43839
--webserver_port=45197
--webserver_interface=127.5.11.195
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:38709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.5.11.195
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 22:27:19 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:19 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
W20260808 22:27:19.065021 20319 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:19.054526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:48989 (local address 127.5.11.254:43839)
0808 22:27:19.054662 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:19.054668 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:19.055718 (+ 1050us) server_negotiation.cc:407] Connection header received
0808 22:27:19.056581 (+ 863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:19.056588 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:19.056676 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:19.056780 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:19.058289 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:19.058857 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:19.060738 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:19.060959 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:19.061065 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:19.064850 (+ 3785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.194:48989: BlockingRecv error: recv got EOF from 127.5.11.194:48989 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 22:27:19.065441 20531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:19.054858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.194:48989)
0808 22:27:19.055553 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:19.055603 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:19.056383 (+ 780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:19.056990 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:19.056999 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:19.057473 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:19.058132 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:19.058153 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:19.058990 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:19.058996 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:19.060576 (+ 1580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:19.060592 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:19.061136 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:19.061143 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:19.061407 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:19.061996 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:19.062023 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:19.064659 (+ 2636us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":124,"threads_started":1}
W20260808 22:27:19.066036 20528 heartbeater.cc:644] Failed to heartbeat to 127.5.11.254:43839 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.11.254:43839: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Aug 08 22:27:19 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:19 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:19.071395 20531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:19.066477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.194:37273)
0808 22:27:19.066589 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:19.066604 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:19.066677 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:19.066932 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:19.066935 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:19.067102 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:19.067187 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:19.067192 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:19.067957 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:19.067960 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:19.068518 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:19.068527 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:19.068973 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:19.068977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:19.069080 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:19.069533 (+ 453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:19.069545 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:19.071266 (+ 1721us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 22:27:19.071596 20319 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:19.066583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:37273 (local address 127.5.11.254:43839)
0808 22:27:19.066694 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:19.066700 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:19.066715 (+ 15us) server_negotiation.cc:407] Connection header received
0808 22:27:19.066756 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:19.066759 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:19.066827 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:19.066915 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:19.067306 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:19.067849 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:19.068645 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:19.068860 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:19.068969 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:19.071378 (+ 2409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.194:37273: BlockingRecv error: recv got EOF from 127.5.11.194:37273 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 08 22:27:19 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:19 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:19.076661 20531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:19.071984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.194:58745)
0808 22:27:19.072080 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:19.072094 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:19.072163 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:19.072366 (+ 203us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:19.072367 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:19.072483 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:19.072546 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:19.072550 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:19.073526 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:19.073528 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:19.073986 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:19.073993 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:19.074537 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:19.074539 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:19.074591 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:19.075019 (+ 428us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:19.075031 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:19.076587 (+ 1556us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 22:27:19.076828 20319 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:19.072049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:58745 (local address 127.5.11.254:43839)
0808 22:27:19.072151 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:19.072155 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:19.072169 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:27:19.072228 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:19.072231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:19.072268 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:19.072360 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:19.072645 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:19.073412 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:19.074099 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:19.074432 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:19.074530 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:19.076725 (+ 2195us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.194:58745: BlockingRecv error: recv got EOF from 127.5.11.194:58745 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260808 22:27:19.077006 20528 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 22:27:19.170145 20533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:19.170373 20533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:19.170439 20533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:19.174360 20533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:19.174412 20533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:19.174510 20533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.195
I20260808 22:27:19.179651 20533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:38709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/krb5kdc/oryx_127.5.11.195.keytab
--principal=oryx/127.5.11.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.195:46677
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.11.195
--webserver_port=45197
--tserver_master_addrs=127.5.11.254:43839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.195
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:19.180631 20533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:19.181607 20533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:19.191048 20539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:19.191277 20533 server_base.cc:1034] running on GCE node
W20260808 22:27:19.191048 20542 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:19.191056 20545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:19.191915 20533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:19.192507 20533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:19.193689 20533 hybrid_clock.cc:648] HybridClock initialized: now 1786228039193657 us; error 43 us; skew 500 ppm
Aug 08 22:27:19 dist-test-slave-7f01 krb5kdc[19127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228039, etypes {rep=17 tkt=17 ses=17}, oryx/127.5.11.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:27:19.196497 20533 init.cc:377] Logged in from keytab as oryx/127.5.11.195@KRBTEST.COM (short username oryx)
I20260808 22:27:19.197350 20533 webserver.cc:460] Webserver started at http://127.5.11.195:45197/ using document root <none> and password file <none>
I20260808 22:27:19.197891 20533 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:19.197989 20533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:19.201789 20533 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.004s
I20260808 22:27:19.203852 20554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:19.205076 20533 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 22:27:19.205211 20533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-2/wal
uuid: "a6be3d39bfc847d399c62c4ea378fd17"
format_stamp: "Formatted at 2026-08-08 22:27:18 on dist-test-slave-7f01"
server_key: "5eddac08fc0b8ebff4b16c4a364083e3"
server_key_iv: "4149427c577c62eb2608bc1a13ae1de4"
server_key_version: "encryptionkey@0"
I20260808 22:27:19.205727 20533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:19.222481 20533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:19.225281 20533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:19.225479 20533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:19.226035 20533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:27:19.226862 20533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:27:19.226936 20533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:19.227008 20533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:27:19.227062 20533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:19.236372 20533 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.195:46677
I20260808 22:27:19.236402 20725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.195:46677 every 8 connection(s)
I20260808 22:27:19.237440 20533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
I20260808 22:27:19.242882 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 20533
Aug 08 22:27:19 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:19 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
W20260808 22:27:19.248504 20319 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:19.239130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:52389 (local address 127.5.11.254:43839)
0808 22:27:19.239275 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:19.239279 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:19.240312 (+ 1033us) server_negotiation.cc:407] Connection header received
0808 22:27:19.241213 (+ 901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:19.241217 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:19.241277 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:19.241388 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:19.242556 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:19.243160 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:19.244608 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:19.244833 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:19.244907 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:19.248389 (+ 3482us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.195:52389: BlockingRecv error: recv got EOF from 127.5.11.195:52389 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 22:27:19.249181 20730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:19.239523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.195:52389)
0808 22:27:19.240167 (+ 644us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:19.240208 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:19.240987 (+ 779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:19.241589 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:19.241596 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:19.241931 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:19.242393 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:19.242406 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:19.243289 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:19.243294 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:19.244476 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:19.244488 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:19.244984 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:19.244989 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:19.245207 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:19.245792 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:19.245808 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:19.248302 (+ 2494us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":101,"threads_started":1}
W20260808 22:27:19.249801 20726 heartbeater.cc:644] Failed to heartbeat to 127.5.11.254:43839 (0 consecutive failures): Not authorized: Failed to ping master at 127.5.11.254:43839: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
I20260808 22:27:19.252871 20732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:19.243910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43052 (local address 127.5.11.254:43839)
0808 22:27:19.244127 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:19.244130 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:19.244141 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:27:19.244173 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:19.244175 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:19.244210 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:19.244283 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:19.245164 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:19.245639 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:19.246524 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:19.246838 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:19.246932 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:19.248384 (+ 1452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:19.248390 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:19.248403 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:19.248428 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:19.250684 (+ 2256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:19.251143 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:19.251148 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:19.251152 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:19.251269 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:19.251567 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:19.251571 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:19.251573 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:19.251969 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:19.252179 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:19.252450 (+ 271us) server_negotiation.cc:299] Negotiation successful
0808 22:27:19.252710 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":97,"threads_started":1}
Aug 08 22:27:19 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:19 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:19.255802 20730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:19.250246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.195:35691)
0808 22:27:19.250507 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:19.250519 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:19.250605 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:19.250827 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:19.250830 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:19.250989 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:19.251075 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:19.251081 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:19.252007 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:19.252011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:19.252572 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:19.252583 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:19.253215 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:19.253219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:19.253312 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:19.253806 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:19.253817 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:19.255649 (+ 1832us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189}
W20260808 22:27:19.255868 20319 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:19.250384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:35691 (local address 127.5.11.254:43839)
0808 22:27:19.250476 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:19.250480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:19.250579 (+ 99us) server_negotiation.cc:407] Connection header received
0808 22:27:19.250679 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:19.250683 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:19.250726 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:19.250813 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:19.251191 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:19.251888 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:19.252700 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:19.253088 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:19.253222 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:19.255767 (+ 2545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.195:35691: BlockingRecv error: recv got EOF from 127.5.11.195:35691 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
Aug 08 22:27:19 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:19 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:19.260468 20730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:19.256264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.195:42323)
0808 22:27:19.256369 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:19.256384 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:19.256454 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:19.256670 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:19.256671 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:19.256790 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:19.256852 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:19.256856 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:19.257478 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:19.257480 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:19.257953 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:19.257959 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:19.258334 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:19.258337 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:19.258389 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:19.258743 (+ 354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:19.258754 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:19.260392 (+ 1638us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 22:27:19.260604 20319 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:19.256350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:42323 (local address 127.5.11.254:43839)
0808 22:27:19.256479 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:19.256483 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:19.256503 (+ 20us) server_negotiation.cc:407] Connection header received
0808 22:27:19.256536 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:19.256538 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:19.256578 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:19.256640 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:19.256943 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:19.257391 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:19.258054 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:19.258244 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:19.258315 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:19.260501 (+ 2186us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.195:42323: BlockingRecv error: recv got EOF from 127.5.11.195:42323 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260808 22:27:19.260721 20726 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 22:27:19 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:19 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:19.894054 20733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:19.888208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.193:42235)
0808 22:27:19.888500 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:19.888514 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:19.888604 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:19.888826 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:19.888828 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:19.889015 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:19.889114 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:19.889119 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:19.889845 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:19.889847 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:19.890544 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:19.890555 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:19.891023 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:19.891025 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:19.891093 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:19.891710 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:19.891722 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:19.893851 (+ 2129us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":91,"threads_started":1}
W20260808 22:27:19.894127 20734 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:19.888290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:42235 (local address 127.5.11.254:43839)
0808 22:27:19.888570 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:19.888574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:19.888587 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:27:19.888673 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:19.888675 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:19.888716 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:19.888790 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:19.889216 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:19.889748 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:19.890645 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:19.890863 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:19.891009 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:19.893985 (+ 2976us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.193:42235: BlockingRecv error: recv got EOF from 127.5.11.193:42235 (error 108)
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":67,"threads_started":1}
Aug 08 22:27:20 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:20 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:20.083104 20737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:20.077628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.194:43539)
0808 22:27:20.077892 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:20.077905 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:20.077999 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:20.078256 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:20.078258 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:20.078435 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:20.078530 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:20.078535 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:20.079296 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:20.079298 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:20.079926 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:20.079937 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:20.080360 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:20.080363 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:20.080425 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:20.080912 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:20.080923 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:20.082983 (+ 2060us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":81,"threads_started":1}
W20260808 22:27:20.083228 20734 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:20.077721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:43539 (local address 127.5.11.254:43839)
0808 22:27:20.077907 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:20.077911 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:20.077975 (+ 64us) server_negotiation.cc:407] Connection header received
0808 22:27:20.078088 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:20.078090 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:20.078150 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:20.078216 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:20.078641 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:20.079199 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:20.080042 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:20.080259 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:20.080351 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:20.083111 (+ 2760us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.194:43539: BlockingRecv error: recv got EOF from 127.5.11.194:43539 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 08 22:27:20 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:20 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:20.267624 20739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:20.261335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.195:59807)
0808 22:27:20.261606 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:20.261621 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:20.261726 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:20.262028 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:20.262030 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:20.262211 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:20.262307 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:20.262312 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:20.263442 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:20.263445 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:20.264104 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:20.264117 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:20.264790 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:20.264795 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:20.264893 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:20.265569 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:20.265584 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:20.267505 (+ 1921us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":100,"threads_started":1}
W20260808 22:27:20.267879 20734 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:20.261434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:59807 (local address 127.5.11.254:43839)
0808 22:27:20.261603 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:20.261609 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:20.261714 (+ 105us) server_negotiation.cc:407] Connection header received
0808 22:27:20.261817 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:20.261822 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:20.261895 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:20.262001 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:20.262424 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:20.263324 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:20.264237 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:20.264670 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:20.264795 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:20.267708 (+ 2913us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.195:59807: BlockingRecv error: recv got EOF from 127.5.11.195:59807 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 08 22:27:20 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:20 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:20.900625 20740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:20.894871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.193:38211)
0808 22:27:20.895183 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:20.895198 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:20.895292 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:20.895555 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:20.895557 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:20.895772 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:20.895937 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:20.895945 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:20.896704 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:20.896706 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:20.897381 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:20.897393 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:20.897927 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:20.897930 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:20.897997 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:20.898522 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:20.898535 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:20.900475 (+ 1940us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":91,"threads_started":1}
W20260808 22:27:20.900811 20741 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:20.894965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:38211 (local address 127.5.11.254:43839)
0808 22:27:20.895258 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:20.895261 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:20.895274 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:27:20.895385 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:20.895387 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:20.895432 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:20.895508 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:20.896072 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:20.896607 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:20.897483 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:20.897791 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:20.897913 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:20.900627 (+ 2714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.193:38211: BlockingRecv error: recv got EOF from 127.5.11.193:38211 (error 108)
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":121,"threads_started":1}
Aug 08 22:27:21 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:21 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:21.089555 20742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:21.083959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.194:57827)
0808 22:27:21.084274 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:21.084293 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:21.084422 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:21.084771 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:21.084774 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:21.084998 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:21.085098 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:21.085104 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:21.085881 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:21.085886 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:21.086450 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:21.086459 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:21.086960 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:21.086964 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:21.087054 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:21.087580 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:21.087591 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:21.089410 (+ 1819us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":116,"threads_started":1}
W20260808 22:27:21.089794 20741 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:21.084038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:57827 (local address 127.5.11.254:43839)
0808 22:27:21.084216 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:21.084221 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:21.084369 (+ 148us) server_negotiation.cc:407] Connection header received
0808 22:27:21.084534 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:21.084540 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:21.084599 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:21.084706 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:21.085231 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:21.085767 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:21.086588 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:21.086834 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:21.086945 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:21.089560 (+ 2615us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.194:57827: BlockingRecv error: recv got EOF from 127.5.11.194:57827 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 08 22:27:21 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:21 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:21.273624 20745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:21.268436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.195:44769)
0808 22:27:21.268745 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:21.268759 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:21.268857 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:21.269097 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:21.269100 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:21.269285 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:21.269381 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:21.269386 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:21.270107 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:21.270109 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:21.270656 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:21.270665 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:21.271077 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:21.271080 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:21.271137 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:21.271632 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:21.271644 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:21.273465 (+ 1821us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":112,"threads_started":1}
W20260808 22:27:21.273689 20741 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:21.268581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:44769 (local address 127.5.11.254:43839)
0808 22:27:21.268724 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:21.268727 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:21.268827 (+ 100us) server_negotiation.cc:407] Connection header received
0808 22:27:21.268939 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:21.268941 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:21.268988 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:21.269056 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:21.269482 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:21.270013 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:21.270757 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:21.270981 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:21.271056 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:21.273597 (+ 2541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.195:44769: BlockingRecv error: recv got EOF from 127.5.11.195:44769 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 22:27:21 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:21 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:21.908089 20746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:21.901527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.193:60817)
0808 22:27:21.901827 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:21.901842 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:21.901945 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:21.902195 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:21.902197 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:21.902393 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:21.902517 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:21.902523 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:21.904052 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:21.904056 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:21.904620 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:21.904630 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:21.905136 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:21.905140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:21.905226 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:21.905840 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:21.905853 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:21.907895 (+ 2042us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":75,"threads_started":1}
W20260808 22:27:21.908151 20747 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:21.901581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:60817 (local address 127.5.11.254:43839)
0808 22:27:21.901860 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:21.901864 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:21.901893 (+ 29us) server_negotiation.cc:407] Connection header received
0808 22:27:21.902033 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:21.902036 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:21.902081 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:21.902158 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:21.902669 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:21.903917 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:21.904752 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:21.905006 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:21.905127 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:21.908034 (+ 2907us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.193:60817: BlockingRecv error: recv got EOF from 127.5.11.193:60817 (error 108)
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":138,"threads_started":1}
Aug 08 22:27:22 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:22 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:22.096359 20749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:22.090323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.194:55559)
0808 22:27:22.090658 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:22.090677 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:22.090785 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:22.091076 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:22.091079 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:22.091263 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:22.091375 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:22.091380 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:22.092261 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:22.092263 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:22.092852 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:22.092861 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:22.093269 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:22.093271 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:22.093343 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:22.093924 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:22.093937 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:22.096200 (+ 2263us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":128,"threads_started":1}
W20260808 22:27:22.096536 20747 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:22.090391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:55559 (local address 127.5.11.254:43839)
0808 22:27:22.090574 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:22.090580 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:22.090753 (+ 173us) server_negotiation.cc:407] Connection header received
0808 22:27:22.090862 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:22.090864 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:22.090911 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:22.091024 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:22.091485 (+ 461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:22.092146 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:22.092955 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:22.093171 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:22.093248 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:22.096366 (+ 3118us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.194:55559: BlockingRecv error: recv got EOF from 127.5.11.194:55559 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 08 22:27:22 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:22 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:22.280007 20750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:22.274507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.195:44181)
0808 22:27:22.274790 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:22.274804 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:22.274915 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:22.275177 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:22.275179 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:22.275368 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:22.275462 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:22.275468 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:22.276255 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:22.276258 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:22.276803 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:22.276811 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:22.277237 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:22.277239 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:22.277299 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:22.277879 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:22.277891 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:22.279823 (+ 1932us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":77,"threads_started":1}
W20260808 22:27:22.280055 20747 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:22.274675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:44181 (local address 127.5.11.254:43839)
0808 22:27:22.274820 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:22.274824 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:22.274864 (+ 40us) server_negotiation.cc:407] Connection header received
0808 22:27:22.275003 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:22.275005 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:22.275062 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:22.275130 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:22.275563 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:22.276161 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:22.276914 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:22.277137 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:22.277217 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:22.279928 (+ 2711us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.195:44181: BlockingRecv error: recv got EOF from 127.5.11.195:44181 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 08 22:27:22 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:22 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:22.917948 20751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:22.909031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.193:59967)
0808 22:27:22.909307 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:22.909321 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:22.909413 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:22.909635 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:22.909637 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:22.909818 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:22.909994 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:22.910006 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:22.910751 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:22.910753 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:22.911430 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:22.911439 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:22.912087 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:22.912091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:22.912180 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:22.915858 (+ 3678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:22.915870 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:22.917771 (+ 1901us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":94,"threads_started":1}
W20260808 22:27:22.918022 20752 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:22.909094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:59967 (local address 127.5.11.254:43839)
0808 22:27:22.909352 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:22.909356 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:22.909369 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:27:22.909502 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:22.909504 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:22.909548 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:22.909625 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:22.910122 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:22.910654 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:22.911556 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:22.911964 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:22.912083 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:22.917896 (+ 5813us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.193:59967: BlockingRecv error: recv got EOF from 127.5.11.193:59967 (error 108)
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":67,"threads_started":1}
Aug 08 22:27:23 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:23 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:23.104385 20753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:23.097101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.194:40983)
0808 22:27:23.097393 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:23.097407 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:23.097503 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:23.097807 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:23.097813 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:23.098081 (+ 268us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:23.098221 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:23.098229 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:23.099414 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:23.099420 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:23.100146 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:23.100157 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:23.100816 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:23.100821 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:23.100942 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:23.101622 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:23.101636 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:23.104209 (+ 2573us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":120,"threads_started":1}
W20260808 22:27:23.104442 20752 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:23.097147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:40983 (local address 127.5.11.254:43839)
0808 22:27:23.097310 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:23.097313 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:23.097475 (+ 162us) server_negotiation.cc:407] Connection header received
0808 22:27:23.097603 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:23.097609 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:23.097673 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:23.097763 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:23.098357 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:23.099274 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:23.100278 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:23.100674 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:23.100800 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:23.104335 (+ 3535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.194:40983: BlockingRecv error: recv got EOF from 127.5.11.194:40983 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 08 22:27:23 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:23 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:23.287413 20756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:23.280894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.195:46421)
0808 22:27:23.281251 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:23.281271 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:23.281375 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:23.281641 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:23.281643 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:23.281821 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:23.281916 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:23.281920 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:23.283162 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:23.283165 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:23.283706 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:23.283714 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:23.284440 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:23.284444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:23.284515 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:23.285075 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:23.285087 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:23.287295 (+ 2208us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":81,"threads_started":1}
W20260808 22:27:23.287662 20752 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:23.281002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:46421 (local address 127.5.11.254:43839)
0808 22:27:23.281162 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:23.281168 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:23.281355 (+ 187us) server_negotiation.cc:407] Connection header received
0808 22:27:23.281447 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:23.281453 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:23.281517 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:23.281608 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:23.282034 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:23.283051 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:23.283884 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:23.284317 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:23.284451 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:23.287503 (+ 3052us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.195:46421: BlockingRecv error: recv got EOF from 127.5.11.195:46421 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 22:27:23 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:23 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:23.924489 20758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:23.918788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.193:52519)
0808 22:27:23.919074 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:23.919088 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:23.919183 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:23.919427 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:23.919429 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:23.919618 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:23.919718 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:23.919723 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:23.920500 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:23.920503 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:23.921068 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:23.921077 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:23.921496 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:23.921498 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:23.921618 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:23.922247 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:23.922262 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:23.924304 (+ 2042us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":102,"threads_started":1}
W20260808 22:27:23.924593 20759 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:23.918866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:52519 (local address 127.5.11.254:43839)
0808 22:27:23.919102 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:23.919105 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:23.919138 (+ 33us) server_negotiation.cc:407] Connection header received
0808 22:27:23.919264 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:23.919266 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:23.919309 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:23.919394 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:23.919879 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:23.920402 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:23.921176 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:23.921397 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:23.921474 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:23.924445 (+ 2971us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.193:52519: BlockingRecv error: recv got EOF from 127.5.11.193:52519 (error 108)
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":90,"threads_started":1}
Aug 08 22:27:24 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:24 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:24.111388 20761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:24.105429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.194:54473)
0808 22:27:24.105736 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:24.105751 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:24.105923 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:24.106217 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:24.106220 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:24.106455 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:24.106553 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:24.106558 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:24.107345 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:24.107350 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:24.108022 (+ 672us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:24.108031 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:24.108509 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:24.108513 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:24.108602 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:24.109213 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:24.109225 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:24.111209 (+ 1984us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":92,"threads_started":1}
W20260808 22:27:24.111486 20759 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:24.105551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:54473 (local address 127.5.11.254:43839)
0808 22:27:24.105696 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:24.105701 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:24.105868 (+ 167us) server_negotiation.cc:407] Connection header received
0808 22:27:24.106034 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:24.106039 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:24.106095 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:24.106190 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:24.106680 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:24.107226 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:24.108149 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:24.108387 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:24.108504 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:24.111363 (+ 2859us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.194:54473: BlockingRecv error: recv got EOF from 127.5.11.194:54473 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 22:27:24 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:24 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:24.293874 20764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:24.288266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.195:59361)
0808 22:27:24.288533 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:24.288547 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:24.288640 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:24.288852 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:24.288854 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:24.289048 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:24.289150 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:24.289154 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:24.290035 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:24.290038 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:24.290604 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:24.290612 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:24.291117 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:24.291121 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:24.291188 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:24.291704 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:24.291716 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:24.293725 (+ 2009us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":92,"threads_started":1}
W20260808 22:27:24.294003 20759 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:24.288319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:59361 (local address 127.5.11.254:43839)
0808 22:27:24.288452 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:24.288455 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:24.288609 (+ 154us) server_negotiation.cc:407] Connection header received
0808 22:27:24.288712 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:24.288713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:24.288756 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:24.288842 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:24.289298 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:24.289927 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:24.290728 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:24.291026 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:24.291147 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:24.293863 (+ 2716us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.195:59361: BlockingRecv error: recv got EOF from 127.5.11.195:59361 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 22:27:24 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:24 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:24.931152 20765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:24.925272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.193:39727)
0808 22:27:24.925545 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:24.925559 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:24.925695 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:24.925959 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:24.925962 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:24.926162 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:24.926261 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:24.926266 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:24.927044 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:24.927046 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:24.927607 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:24.927615 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:24.928123 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:24.928126 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:24.928226 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:24.928789 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:24.928801 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:24.930969 (+ 2168us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":93,"threads_started":1}
W20260808 22:27:24.931211 20766 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:24.925322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:39727 (local address 127.5.11.254:43839)
0808 22:27:24.925576 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:24.925579 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:24.925640 (+ 61us) server_negotiation.cc:407] Connection header received
0808 22:27:24.925796 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:24.925798 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:24.925845 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:24.925926 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:24.926380 (+ 454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:24.926946 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:24.927751 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:24.927996 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:24.928111 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:24.931093 (+ 2982us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.193:39727: BlockingRecv error: recv got EOF from 127.5.11.193:39727 (error 108)
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":102,"threads_started":1}
Aug 08 22:27:25 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:25 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:25.118237 20767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:25.112269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.194:34929)
0808 22:27:25.112556 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:25.112571 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:25.112683 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:25.112938 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:25.112940 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:25.113142 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:25.113241 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:25.113246 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:25.114178 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:25.114182 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:25.114957 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:25.114967 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:25.115421 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:25.115424 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:25.115487 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:25.116011 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:25.116032 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:25.118066 (+ 2034us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":97,"threads_started":1}
W20260808 22:27:25.118288 20766 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:25.112354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:34929 (local address 127.5.11.254:43839)
0808 22:27:25.112525 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:25.112531 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:25.112674 (+ 143us) server_negotiation.cc:407] Connection header received
0808 22:27:25.112771 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:25.112774 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:25.112823 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:25.112888 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:25.113381 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:25.114040 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:25.115089 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:25.115319 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:25.115401 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:25.118185 (+ 2784us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.194:34929: BlockingRecv error: recv got EOF from 127.5.11.194:34929 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 08 22:27:25 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:25 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:25.300247 20768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:25.294612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.195:56443)
0808 22:27:25.294965 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:25.294980 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:25.295092 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:25.295362 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:25.295365 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:25.295561 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:25.295660 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:25.295666 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:25.296488 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:25.296491 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:25.297105 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:25.297116 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:25.297636 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:25.297642 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:25.297727 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:25.298219 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:25.298230 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:25.300126 (+ 1896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":97,"threads_started":1}
W20260808 22:27:25.300436 20766 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:25.294724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:56443 (local address 127.5.11.254:43839)
0808 22:27:25.294884 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:25.294890 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:25.295060 (+ 170us) server_negotiation.cc:407] Connection header received
0808 22:27:25.295175 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:25.295177 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:25.295224 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:25.295291 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:25.295852 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:25.296379 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:25.297224 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:25.297531 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:25.297611 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:25.300266 (+ 2655us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.195:56443: BlockingRecv error: recv got EOF from 127.5.11.195:56443 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 08 22:27:25 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:25 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:25.937716 20770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:25.932040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.193:47191)
0808 22:27:25.932395 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:25.932414 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:25.932571 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:25.932839 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:25.932842 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:25.933047 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:25.933154 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:25.933160 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:25.933897 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:25.933899 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:25.934511 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:25.934519 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:25.934989 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:25.934993 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:25.935066 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:25.935669 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:25.935690 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:25.937599 (+ 1909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":94,"threads_started":1}
W20260808 22:27:25.937948 20769 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:25.932045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:47191 (local address 127.5.11.254:43839)
0808 22:27:25.932379 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:25.932385 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:25.932535 (+ 150us) server_negotiation.cc:407] Connection header received
0808 22:27:25.932669 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:25.932672 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:25.932719 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:25.932796 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:25.933263 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:25.933798 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:25.934615 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:25.934837 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:25.934918 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:25.937783 (+ 2865us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.193:47191: BlockingRecv error: recv got EOF from 127.5.11.193:47191 (error 108)
Metrics: {"server-negotiator.queue_time_us":229,"thread_start_us":97,"threads_started":1}
Aug 08 22:27:26 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:26 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:26.124562 20775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:26.119086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.194:44643)
0808 22:27:26.119401 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:26.119415 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:26.119513 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:26.119757 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:26.119759 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:26.119953 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:26.120047 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:26.120052 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:26.120748 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:26.120751 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:26.121289 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:26.121296 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:26.121698 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:26.121700 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:26.121760 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:26.122253 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:26.122264 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:26.124387 (+ 2123us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":91,"threads_started":1}
W20260808 22:27:26.124604 20769 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:26.119209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:44643 (local address 127.5.11.254:43839)
0808 22:27:26.119383 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:26.119387 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:26.119485 (+ 98us) server_negotiation.cc:407] Connection header received
0808 22:27:26.119579 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:26.119581 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:26.119626 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:26.119698 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:26.120146 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:26.120658 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:26.121395 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:26.121604 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:26.121677 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:26.124503 (+ 2826us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.194:44643: BlockingRecv error: recv got EOF from 127.5.11.194:44643 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 08 22:27:26 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:26 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:26.307178 20777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:26.301164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.195:37701)
0808 22:27:26.301615 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:26.301627 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:26.301771 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:26.302069 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:26.302072 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:26.302297 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:26.302388 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:26.302401 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:26.303179 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:26.303184 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:26.303774 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:26.303784 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:26.304255 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:26.304259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:26.304345 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:26.304834 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:26.304846 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:26.307002 (+ 2156us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":96,"threads_started":1}
W20260808 22:27:26.307224 20769 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:26.301231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:37701 (local address 127.5.11.254:43839)
0808 22:27:26.301494 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:26.301501 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:26.301709 (+ 208us) server_negotiation.cc:407] Connection header received
0808 22:27:26.301876 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:26.301879 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:26.301953 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:26.302041 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:26.302521 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:26.303068 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:26.303906 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:26.304139 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:26.304253 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:26.307123 (+ 2870us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.195:37701: BlockingRecv error: recv got EOF from 127.5.11.195:37701 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 08 22:27:26 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:26 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:26.943662 20778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:26.938582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.193:58271)
0808 22:27:26.938850 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:26.938864 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:26.938951 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:26.939180 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:26.939182 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:26.939363 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:26.939462 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:26.939467 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:26.940258 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:26.940261 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:26.940805 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:26.940813 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:26.941209 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:26.941211 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:26.941272 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:26.941746 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:26.941757 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:26.943545 (+ 1788us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":96,"threads_started":1}
W20260808 22:27:26.943869 20779 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:26.938674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:58271 (local address 127.5.11.254:43839)
0808 22:27:26.938907 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:26.938910 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:26.938923 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:27:26.939027 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:26.939029 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:26.939071 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:26.939146 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:26.939561 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:26.940154 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:26.940900 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:26.941115 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:26.941188 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:26.943676 (+ 2488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.193:58271: BlockingRecv error: recv got EOF from 127.5.11.193:58271 (error 108)
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":94,"threads_started":1}
Aug 08 22:27:27 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:27 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:27.131000 20780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:27.125322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.194:47393)
0808 22:27:27.125666 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:27.125689 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:27.125788 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:27.126029 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:27.126032 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:27.126218 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:27.126317 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:27.126322 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:27.127185 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:27.127187 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:27.127728 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:27.127793 (+ 65us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:27.128217 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:27.128220 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:27.128282 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:27.128765 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:27.128776 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:27.130804 (+ 2028us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":172,"threads_started":1}
W20260808 22:27:27.131111 20779 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:27.125401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:47393 (local address 127.5.11.254:43839)
0808 22:27:27.125547 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:27.125551 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:27.125756 (+ 205us) server_negotiation.cc:407] Connection header received
0808 22:27:27.125866 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:27.125868 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:27.125912 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:27.125983 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:27.126414 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:27.127091 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:27.127899 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:27.128122 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:27.128199 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:27.130938 (+ 2739us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.194:47393: BlockingRecv error: recv got EOF from 127.5.11.194:47393 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 08 22:27:27 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:27 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:27.314142 20782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:27.308134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.195:33301)
0808 22:27:27.308432 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:27.308447 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:27.308604 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:27.308868 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:27.308871 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:27.309113 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:27.309211 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:27.309216 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:27.310123 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:27.310129 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:27.310697 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:27.310706 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:27.311204 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:27.311207 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:27.311293 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:27.311919 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:27.311942 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:27.314025 (+ 2083us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":93,"threads_started":1}
W20260808 22:27:27.314276 20779 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:27.308203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:33301 (local address 127.5.11.254:43839)
0808 22:27:27.308375 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:27.308382 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:27.308541 (+ 159us) server_negotiation.cc:407] Connection header received
0808 22:27:27.308696 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:27.308700 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:27.308756 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:27.308846 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:27.309353 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:27.310016 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:27.310853 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:27.311085 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:27.311200 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:27.314155 (+ 2955us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.195:33301: BlockingRecv error: recv got EOF from 127.5.11.195:33301 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 08 22:27:27 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:27 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:27.953464 20784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:27.944424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.193:53377)
0808 22:27:27.944706 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:27.944720 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:27.944814 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:27.945030 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:27.945032 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:27.945211 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:27.945307 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:27.945312 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:27.946036 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:27.946038 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:27.946590 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:27.946597 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:27.947023 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:27.947025 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:27.947087 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:27.951316 (+ 4229us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:27.951328 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:27.953325 (+ 1997us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":94,"threads_started":1}
W20260808 22:27:27.953620 20785 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:27.944475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:53377 (local address 127.5.11.254:43839)
0808 22:27:27.944720 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:27.944724 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:27.944768 (+ 44us) server_negotiation.cc:407] Connection header received
0808 22:27:27.944878 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:27.944879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:27.944947 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:27.945018 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:27.945405 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:27.945942 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:27.946689 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:27.946917 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:27.947001 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:27.953461 (+ 6460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.193:53377: BlockingRecv error: recv got EOF from 127.5.11.193:53377 (error 108)
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":71,"threads_started":1}
Aug 08 22:27:28 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:28 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:28.137822 20788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:28.131803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.194:37339)
0808 22:27:28.132355 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:28.132368 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:28.132522 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:28.132791 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:28.132794 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:28.133006 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:28.133100 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:28.133106 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:28.133860 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:28.133866 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:28.134417 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:28.134425 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:28.134889 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:28.134893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:28.134968 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:28.135544 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:28.135556 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:28.137621 (+ 2065us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":138,"threads_started":1}
W20260808 22:27:28.137843 20785 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:28.131950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:37339 (local address 127.5.11.254:43839)
0808 22:27:28.132121 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:28.132126 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:28.132459 (+ 333us) server_negotiation.cc:407] Connection header received
0808 22:27:28.132625 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:28.132628 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:28.132684 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:28.132770 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:28.133221 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:28.133754 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:28.134550 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:28.134778 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:28.134885 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:28.137743 (+ 2858us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.194:37339: BlockingRecv error: recv got EOF from 127.5.11.194:37339 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 08 22:27:28 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:28 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
W20260808 22:27:28.320559 20785 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:28.315131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:43713 (local address 127.5.11.254:43839)
0808 22:27:28.315258 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:28.315262 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:28.315446 (+ 184us) server_negotiation.cc:407] Connection header received
0808 22:27:28.315568 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:28.315570 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:28.315618 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:28.315685 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:28.316185 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:28.316757 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:28.317490 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:28.317698 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:28.317776 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:28.320462 (+ 2686us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.195:43713: BlockingRecv error: recv got EOF from 127.5.11.195:43713 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 22:27:28.320551 20790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:28.315065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.195:43713)
0808 22:27:28.315363 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:28.315376 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:28.315473 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:28.315726 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:28.315728 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:28.315970 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:28.316069 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:28.316075 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:28.316854 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:28.316856 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:28.317392 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:28.317400 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:28.317797 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:28.317800 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:28.317863 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:28.318352 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:28.318362 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:28.320341 (+ 1979us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":113,"threads_started":1}
Aug 08 22:27:28 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:28 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:28.959658 20791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:28.954486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.193:59451)
0808 22:27:28.954755 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:28.954771 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:28.954876 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:28.955124 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:28.955126 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:28.955314 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:28.955411 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:28.955416 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:28.956161 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:28.956163 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:28.956707 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:28.956715 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:28.957114 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:28.957117 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:28.957179 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:28.957673 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:28.957684 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:28.959539 (+ 1855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":96,"threads_started":1}
W20260808 22:27:28.959878 20792 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:28.954527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:59451 (local address 127.5.11.254:43839)
0808 22:27:28.954775 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:28.954779 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:28.954856 (+ 77us) server_negotiation.cc:407] Connection header received
0808 22:27:28.954959 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:28.954961 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:28.955004 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:28.955083 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:28.955525 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:28.956070 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:28.956802 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:28.957018 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:28.957092 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:28.959677 (+ 2585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.193:59451: BlockingRecv error: recv got EOF from 127.5.11.193:59451 (error 108)
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":92,"threads_started":1}
Aug 08 22:27:29 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:29 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:29.143919 20794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:29.138714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.194:56845)
0808 22:27:29.139009 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:29.139022 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:29.139115 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:29.139341 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:29.139343 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:29.139522 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:29.139621 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:29.139626 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:29.140388 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:29.140391 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:29.140942 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:29.140950 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:29.141348 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:29.141351 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:29.141410 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:29.141892 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:29.141903 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:29.143800 (+ 1897us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":102,"threads_started":1}
W20260808 22:27:29.144043 20792 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:29.138795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:56845 (local address 127.5.11.254:43839)
0808 22:27:29.138952 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:29.138959 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:29.139091 (+ 132us) server_negotiation.cc:407] Connection header received
0808 22:27:29.139195 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:29.139197 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:29.139241 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:29.139299 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:29.139719 (+ 420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:29.140287 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:29.141039 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:29.141254 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:29.141328 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:29.143921 (+ 2593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.194:56845: BlockingRecv error: recv got EOF from 127.5.11.194:56845 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 08 22:27:29 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:29 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:29.330457 20795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:29.321376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.195:46399)
0808 22:27:29.321720 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:29.321739 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:29.321892 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:29.322174 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:29.322177 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:29.322399 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:29.322495 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:29.322501 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:29.323295 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:29.323300 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:29.323968 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:29.323978 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:29.324453 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:29.324457 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:29.324535 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:29.328472 (+ 3937us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:29.328484 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:29.330285 (+ 1801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":138,"threads_started":1}
W20260808 22:27:29.330559 20792 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:29.321545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:46399 (local address 127.5.11.254:43839)
0808 22:27:29.321732 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:29.321738 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:29.321828 (+ 90us) server_negotiation.cc:407] Connection header received
0808 22:27:29.321996 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:29.322000 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:29.322052 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:29.322149 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:29.322623 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:29.323178 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:29.324085 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:29.324330 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:29.324451 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:29.330433 (+ 5982us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.195:46399: BlockingRecv error: recv got EOF from 127.5.11.195:46399 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 08 22:27:29 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:29 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:29.966123 20796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:29.960499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.193:54649)
0808 22:27:29.960773 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:29.960787 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:29.960878 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:29.961142 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:29.961144 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:29.961332 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:29.961432 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:29.961437 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:29.962153 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:29.962155 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:29.962772 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:29.962783 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:29.963326 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:29.963330 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:29.963437 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:29.964093 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:29.964112 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:29.966006 (+ 1894us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":90,"threads_started":1}
W20260808 22:27:29.966325 20797 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:29.960563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:54649 (local address 127.5.11.254:43839)
0808 22:27:29.960800 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:29.960804 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:29.960829 (+ 25us) server_negotiation.cc:407] Connection header received
0808 22:27:29.960972 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:29.960974 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:29.961028 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:29.961106 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:29.961533 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:29.962059 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:29.962905 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:29.963197 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:29.963316 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:29.966149 (+ 2833us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.193:54649: BlockingRecv error: recv got EOF from 127.5.11.193:54649 (error 108)
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":68,"threads_started":1}
Aug 08 22:27:30 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:30 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:30.150978 20800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:30.144816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.194:48517)
0808 22:27:30.145095 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:30.145109 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:30.145216 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:30.145441 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:30.145443 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:30.145633 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:30.145733 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:30.145738 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:30.146571 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:30.146575 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:30.147946 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:30.147958 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:30.148389 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:30.148392 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:30.148455 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:30.148947 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:30.148959 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:30.150862 (+ 1903us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":98,"threads_started":1}
W20260808 22:27:30.151134 20797 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:30.144878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:48517 (local address 127.5.11.254:43839)
0808 22:27:30.145044 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:30.145051 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:30.145183 (+ 132us) server_negotiation.cc:407] Connection header received
0808 22:27:30.145291 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:30.145294 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:30.145338 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:30.145405 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:30.145832 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:30.146432 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:30.148060 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:30.148279 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:30.148358 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:30.151010 (+ 2652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.194:48517: BlockingRecv error: recv got EOF from 127.5.11.194:48517 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 08 22:27:30 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:30 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:30.336702 20802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:30.331297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.195:42213)
0808 22:27:30.331575 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:30.331590 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:30.331731 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:30.332009 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:30.332011 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:30.332201 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:30.332297 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:30.332303 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:30.333121 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:30.333124 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:30.333678 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:30.333686 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:30.334094 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:30.334097 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:30.334159 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:30.334658 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:30.334670 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:30.336584 (+ 1914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":103,"threads_started":1}
W20260808 22:27:30.336835 20797 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:30.331386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:42213 (local address 127.5.11.254:43839)
0808 22:27:30.331527 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:30.331532 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:30.331683 (+ 151us) server_negotiation.cc:407] Connection header received
0808 22:27:30.331848 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:30.331850 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:30.331900 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:30.331968 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:30.332417 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:30.333017 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:30.333782 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:30.333996 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:30.334070 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:30.336717 (+ 2647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.195:42213: BlockingRecv error: recv got EOF from 127.5.11.195:42213 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 08 22:27:30 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:30 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:30.972512 20803 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:30.966970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.193:59853)
0808 22:27:30.967254 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:30.967269 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:30.967367 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:30.967623 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:30.967625 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:30.967850 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:30.967952 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:30.967963 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:30.968760 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:30.968762 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:30.969311 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:30.969318 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:30.969732 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:30.969734 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:30.969795 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:30.970473 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:30.970486 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:30.972394 (+ 1908us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":98,"threads_started":1}
W20260808 22:27:30.972712 20804 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:30.967030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:59853 (local address 127.5.11.254:43839)
0808 22:27:30.967274 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:30.967278 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:30.967334 (+ 56us) server_negotiation.cc:407] Connection header received
0808 22:27:30.967457 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:30.967459 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:30.967506 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:30.967586 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:30.968093 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:30.968653 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:30.969407 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:30.969635 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:30.969710 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:30.972537 (+ 2827us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.193:59853: BlockingRecv error: recv got EOF from 127.5.11.193:59853 (error 108)
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":93,"threads_started":1}
Aug 08 22:27:31 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:31 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:31.157793 20806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:31.151954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.194:39203)
0808 22:27:31.152371 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:31.152395 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:31.152561 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:31.152863 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:31.152867 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:31.153085 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:31.153241 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:31.153249 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:31.154072 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:31.154077 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:31.154626 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:31.154634 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:31.155117 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:31.155120 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:31.155204 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:31.155716 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:31.155727 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:31.157680 (+ 1953us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":109,"threads_started":1}
W20260808 22:27:31.157949 20804 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:31.152078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:39203 (local address 127.5.11.254:43839)
0808 22:27:31.152284 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:31.152289 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:31.152511 (+ 222us) server_negotiation.cc:407] Connection header received
0808 22:27:31.152686 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:31.152689 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:31.152753 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:31.152836 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:31.153358 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:31.153968 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:31.154755 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:31.154997 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:31.155102 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:31.157826 (+ 2724us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.194:39203: BlockingRecv error: recv got EOF from 127.5.11.194:39203 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 08 22:27:31 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:31 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:31.343024 20807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:31.337537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.195:48057)
0808 22:27:31.337822 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:31.337843 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:31.337969 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:31.338254 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:31.338257 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:31.338469 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:31.338563 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:31.338568 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:31.339332 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:31.339338 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:31.339987 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:31.339996 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:31.340446 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:31.340450 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:31.340526 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:31.341048 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:31.341061 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:31.342839 (+ 1778us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":89,"threads_started":1}
W20260808 22:27:31.343070 20804 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:31.337608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:48057 (local address 127.5.11.254:43839)
0808 22:27:31.337774 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:31.337779 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:31.337915 (+ 136us) server_negotiation.cc:407] Connection header received
0808 22:27:31.338087 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:31.338091 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:31.338148 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:31.338231 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:31.338684 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:31.339222 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:31.340107 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:31.340333 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:31.340444 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:31.342970 (+ 2526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.195:48057: BlockingRecv error: recv got EOF from 127.5.11.195:48057 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 08 22:27:31 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:31 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:31.979197 20809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:31.973420 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.193:39365)
0808 22:27:31.973734 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:31.973749 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:31.973848 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:31.974114 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:31.974116 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:31.974347 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:31.974453 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:31.974458 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:31.975180 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:31.975182 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:31.975788 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:31.975797 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:31.976202 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:31.976205 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:31.976267 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:31.976844 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:31.976856 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:31.979022 (+ 2166us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":76,"threads_started":1}
W20260808 22:27:31.979351 20811 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:31.973530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:39365 (local address 127.5.11.254:43839)
0808 22:27:31.973803 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:31.973807 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:31.973822 (+ 15us) server_negotiation.cc:407] Connection header received
0808 22:27:31.973948 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:31.973950 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:31.973994 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:31.974069 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:31.974562 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:31.975085 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:31.975890 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:31.976107 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:31.976178 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:31.979190 (+ 3012us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.193:39365: BlockingRecv error: recv got EOF from 127.5.11.193:39365 (error 108)
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":84,"threads_started":1}
Aug 08 22:27:32 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:32 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:32.164294 20812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:32.158666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.194:44341)
0808 22:27:32.158967 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:32.158981 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:32.159083 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:32.159317 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:32.159319 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:32.159521 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:32.159616 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:32.159621 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:32.160377 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:32.160380 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:32.160957 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:32.160965 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:32.161468 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:32.161472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:32.161557 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:32.162155 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:32.162167 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:32.164109 (+ 1942us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":90,"threads_started":1}
W20260808 22:27:32.164354 20811 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:32.158836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:44341 (local address 127.5.11.254:43839)
0808 22:27:32.158987 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:32.158991 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:32.159046 (+ 55us) server_negotiation.cc:407] Connection header received
0808 22:27:32.159165 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:32.159167 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:32.159212 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:32.159277 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:32.159695 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:32.160275 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:32.161090 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:32.161337 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:32.161455 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:32.164240 (+ 2785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.194:44341: BlockingRecv error: recv got EOF from 127.5.11.194:44341 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 08 22:27:32 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:32 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:32.349514 20817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:32.344029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.195:59645)
0808 22:27:32.344332 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:32.344346 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:32.344446 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:32.344653 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:32.344655 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:32.344875 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:32.344973 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:32.344979 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:32.345809 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:32.345812 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:32.346384 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:32.346394 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:32.346837 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:32.346840 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:32.346903 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:32.347371 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:32.347388 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:32.349318 (+ 1930us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":118,"threads_started":1}
W20260808 22:27:32.349539 20811 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:32.344069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:59645 (local address 127.5.11.254:43839)
0808 22:27:32.344215 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:32.344220 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:32.344414 (+ 194us) server_negotiation.cc:407] Connection header received
0808 22:27:32.344511 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:32.344513 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:32.344560 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:32.344627 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:32.345070 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:32.345677 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:32.346495 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:32.346703 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:32.346791 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:32.349438 (+ 2647us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.195:59645: BlockingRecv error: recv got EOF from 127.5.11.195:59645 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 08 22:27:32 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:32 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:32.990012 20818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:32.980164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.193:45567)
0808 22:27:32.980479 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:32.980498 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:32.980610 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:32.980860 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:32.980863 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:32.981057 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:32.981186 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:32.981193 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:32.982285 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:32.982292 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:32.982898 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:32.982908 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:32.983321 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:32.983323 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:32.983386 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:32.987974 (+ 4588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:32.987987 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:32.989872 (+ 1885us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":94,"threads_started":1}
W20260808 22:27:32.990237 20819 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:32.980274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:45567 (local address 127.5.11.254:43839)
0808 22:27:32.980626 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:32.980629 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:32.980643 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:27:32.980678 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:32.980680 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:32.980721 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:32.980798 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:32.981328 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:32.982144 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:32.983008 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:32.983224 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:32.983302 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:32.990059 (+ 6757us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.193:45567: BlockingRecv error: recv got EOF from 127.5.11.193:45567 (error 108)
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":82,"threads_started":1}
Aug 08 22:27:33 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:33 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:33.170739 20820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:33.165117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.194:41051)
0808 22:27:33.165465 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:33.165479 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:33.165616 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:33.165892 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:33.165895 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:33.166113 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:33.166209 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:33.166214 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:33.167020 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:33.167025 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:33.167595 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:33.167603 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:33.168116 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:33.168120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:33.168204 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:33.168726 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:33.168737 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:33.170605 (+ 1868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":116,"threads_started":1}
W20260808 22:27:33.170857 20819 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:33.165182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:41051 (local address 127.5.11.254:43839)
0808 22:27:33.165413 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:33.165419 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:33.165555 (+ 136us) server_negotiation.cc:407] Connection header received
0808 22:27:33.165715 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:33.165720 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:33.165775 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:33.165861 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:33.166336 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:33.166907 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:33.167720 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:33.167995 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:33.168111 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:33.170745 (+ 2634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.194:41051: BlockingRecv error: recv got EOF from 127.5.11.194:41051 (error 108)
Metrics: {"server-negotiator.queue_time_us":121}
Aug 08 22:27:33 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:33 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.195@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:33.355456 20822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:33.350285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.195:40759)
0808 22:27:33.350574 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:33.350587 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:33.350688 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:33.350926 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:33.350928 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:33.351118 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:33.351215 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:33.351221 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:33.351949 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:33.351952 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:33.352531 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:33.352540 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:33.352930 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:33.352933 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:33.352996 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:33.353469 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:33.353481 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:33.355306 (+ 1825us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":90,"threads_started":1}
W20260808 22:27:33.355638 20819 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:33.350412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:40759 (local address 127.5.11.254:43839)
0808 22:27:33.350580 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:33.350584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:33.350656 (+ 72us) server_negotiation.cc:407] Connection header received
0808 22:27:33.350764 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:33.350766 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:33.350813 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:33.350882 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:33.351312 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:33.351848 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:33.352630 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:33.352836 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:33.352910 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:33.355481 (+ 2571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.195:40759: BlockingRecv error: recv got EOF from 127.5.11.195:40759 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 08 22:27:33 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:33 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.193@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:33.996483 20823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:33.991034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.193:52663)
0808 22:27:33.991347 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:33.991361 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:33.991470 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:33.991713 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:33.991715 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:33.991985 (+ 270us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:33.992150 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:33.992170 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:33.992893 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:33.992897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:33.993507 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:33.993517 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:33.993929 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:33.993932 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:33.993994 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:33.994487 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:33.994498 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:33.996343 (+ 1845us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":103,"threads_started":1}
W20260808 22:27:33.996649 20824 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:33.991045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:52663 (local address 127.5.11.254:43839)
0808 22:27:33.991346 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:33.991350 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:33.991425 (+ 75us) server_negotiation.cc:407] Connection header received
0808 22:27:33.991550 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:33.991552 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:33.991596 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:33.991673 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:33.992265 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:33.992795 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:33.993608 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:33.993830 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:33.993909 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:33.996512 (+ 2603us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.193:52663: BlockingRecv error: recv got EOF from 127.5.11.193:52663 (error 108)
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":100,"threads_started":1}
Aug 08 22:27:34 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
Aug 08 22:27:34 dist-test-slave-7f01 krb5kdc[19127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.5.11.194@KRBTEST.COM for oryx/127.5.11.254@KRBTEST.COM, Server not found in Kerberos database
I20260808 22:27:34.176798 20825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:34.171695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43839 (local address 127.5.11.194:42367)
0808 22:27:34.172000 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:34.172020 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:34.172110 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:34.172299 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:34.172301 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:34.172489 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:34.172582 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:34.172587 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:34.173281 (+ 694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:34.173283 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:34.173834 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:34.173841 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:34.174232 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:34.174235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:34.174298 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:34.174747 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:34.174758 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:34.176674 (+ 1916us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.5.11.254:43839: Server oryx/127.5.11.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":120,"threads_started":1}
W20260808 22:27:34.177093 20824 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:34.171811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:42367 (local address 127.5.11.254:43839)
0808 22:27:34.171947 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:34.171950 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:34.172084 (+ 134us) server_negotiation.cc:407] Connection header received
0808 22:27:34.172176 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:34.172178 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:34.172219 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:34.172288 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:34.172670 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:34.173190 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:34.173928 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:34.174140 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:34.174214 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:34.176812 (+ 2598us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.5.11.194:42367: BlockingRecv error: recv got EOF from 127.5.11.194:42367 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 22:27:34.246537 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 20079
I20260808 22:27:34.258776 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 20327
I20260808 22:27:34.270709 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 20533
I20260808 22:27:34.286092 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 19974
2026-08-08T22:27:34Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18234 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-08T22:27:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:27:34Z Disabled control of system clock
I20260808 22:27:34.320976 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:33871
--webserver_interface=127.5.11.254
--webserver_port=0
--builtin_ntp_servers=127.5.11.212:39379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:33871
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260808 22:27:34.432360 20830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:34.432595 20830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:34.432627 20830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:34.436846 20830 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:27:34.436905 20830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:34.436923 20830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:34.436939 20830 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:27:34.436983 20830 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:27:34.441664 20830 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:39379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:33871
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:33871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20830
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:34.442663 20830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:34.444136 20830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:34.452766 20846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:34.452775 20841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:34.452831 20842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:34.452987 20830 server_base.cc:1034] running on GCE node
I20260808 22:27:34.453647 20830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:34.454619 20830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:34.455816 20830 hybrid_clock.cc:648] HybridClock initialized: now 1786228054455781 us; error 38 us; skew 500 ppm
I20260808 22:27:34.457616 20830 webserver.cc:460] Webserver started at http://127.5.11.254:36131/ using document root <none> and password file <none>
I20260808 22:27:34.458170 20830 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:34.458233 20830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:34.458477 20830 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:27:34.460518 20830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "44195cbc2abb441d89c021024447b35c"
format_stamp: "Formatted at 2026-08-08 22:27:34 on dist-test-slave-7f01"
server_key: "cf90f7238f1803f839884ba3c5826022"
server_key_iv: "0ec01e68ae72716796f77260df8ed95e"
server_key_version: "encryptionkey@0"
I20260808 22:27:34.461076 20830 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "44195cbc2abb441d89c021024447b35c"
format_stamp: "Formatted at 2026-08-08 22:27:34 on dist-test-slave-7f01"
server_key: "cf90f7238f1803f839884ba3c5826022"
server_key_iv: "0ec01e68ae72716796f77260df8ed95e"
server_key_version: "encryptionkey@0"
I20260808 22:27:34.464576 20830 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:27:34.466778 20855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:34.467988 20830 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:27:34.468114 20830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "44195cbc2abb441d89c021024447b35c"
format_stamp: "Formatted at 2026-08-08 22:27:34 on dist-test-slave-7f01"
server_key: "cf90f7238f1803f839884ba3c5826022"
server_key_iv: "0ec01e68ae72716796f77260df8ed95e"
server_key_version: "encryptionkey@0"
I20260808 22:27:34.468218 20830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:34.486140 20830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:34.486816 20830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:34.487011 20830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:34.493719 20830 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:33871
I20260808 22:27:34.493726 20939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:33871 every 8 connection(s)
I20260808 22:27:34.494647 20830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:27:34.496479 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 20830
I20260808 22:27:34.496574 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:27:34.496867 5167 external_mini_cluster.cc:1428] Setting key e5badd09a53229d213a26189efa84a08
I20260808 22:27:34.498169 20941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:27:34.504966 20941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44195cbc2abb441d89c021024447b35c: Bootstrap starting.
I20260808 22:27:34.507351 20946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:34.498346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41240 (local address 127.5.11.254:33871)
0808 22:27:34.498998 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:34.499019 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:34.499063 (+ 44us) server_negotiation.cc:407] Connection header received
0808 22:27:34.499976 (+ 913us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:34.500001 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:34.500494 (+ 493us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:34.500861 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:34.501297 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:34.502792 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:34.503618 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:34.504344 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:34.504508 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:34.505312 (+ 804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:34.505318 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:34.505334 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:27:34.505795 (+ 461us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:34.505879 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:27:34.505906 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:34.506007 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:34.506114 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 22:27:34.506403 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":102,"threads_started":1}
I20260808 22:27:34.507678 20941 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44195cbc2abb441d89c021024447b35c: Neither blocks nor log segments found. Creating new log.
I20260808 22:27:34.508500 20941 log.cc:824] T 00000000000000000000000000000000 P 44195cbc2abb441d89c021024447b35c: Log is configured to *not* fsync() on all Append() calls
I20260808 22:27:34.510360 20941 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44195cbc2abb441d89c021024447b35c: No bootstrap required, opened a new log
I20260808 22:27:34.513643 20941 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44195cbc2abb441d89c021024447b35c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44195cbc2abb441d89c021024447b35c" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 33871 } }
I20260808 22:27:34.513859 20941 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44195cbc2abb441d89c021024447b35c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:27:34.513916 20941 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44195cbc2abb441d89c021024447b35c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44195cbc2abb441d89c021024447b35c, State: Initialized, Role: FOLLOWER
I20260808 22:27:34.514302 20941 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44195cbc2abb441d89c021024447b35c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44195cbc2abb441d89c021024447b35c" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 33871 } }
I20260808 22:27:34.514410 20941 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44195cbc2abb441d89c021024447b35c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:27:34.514451 20941 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44195cbc2abb441d89c021024447b35c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:27:34.514519 20941 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44195cbc2abb441d89c021024447b35c [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:27:34.515370 20941 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44195cbc2abb441d89c021024447b35c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44195cbc2abb441d89c021024447b35c" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 33871 } }
I20260808 22:27:34.515762 20941 leader_election.cc:302] T 00000000000000000000000000000000 P 44195cbc2abb441d89c021024447b35c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44195cbc2abb441d89c021024447b35c; no voters:
I20260808 22:27:34.516054 20941 leader_election.cc:288] T 00000000000000000000000000000000 P 44195cbc2abb441d89c021024447b35c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:27:34.516173 20949 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44195cbc2abb441d89c021024447b35c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:27:34.516405 20949 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44195cbc2abb441d89c021024447b35c [term 1 LEADER]: Becoming Leader. State: Replica: 44195cbc2abb441d89c021024447b35c, State: Running, Role: LEADER
I20260808 22:27:34.516757 20949 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44195cbc2abb441d89c021024447b35c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44195cbc2abb441d89c021024447b35c" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 33871 } }
I20260808 22:27:34.517271 20941 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44195cbc2abb441d89c021024447b35c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:27:34.518369 20954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44195cbc2abb441d89c021024447b35c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44195cbc2abb441d89c021024447b35c. Latest consensus state: current_term: 1 leader_uuid: "44195cbc2abb441d89c021024447b35c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44195cbc2abb441d89c021024447b35c" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 33871 } } }
I20260808 22:27:34.518517 20954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44195cbc2abb441d89c021024447b35c [sys.catalog]: This master's current role is: LEADER
I20260808 22:27:34.518846 20953 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44195cbc2abb441d89c021024447b35c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44195cbc2abb441d89c021024447b35c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44195cbc2abb441d89c021024447b35c" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 33871 } } }
I20260808 22:27:34.518908 20963 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:27:34.519503 20953 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44195cbc2abb441d89c021024447b35c [sys.catalog]: This master's current role is: LEADER
I20260808 22:27:34.521733 20963 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:27:34.527083 20963 catalog_manager.cc:1347] Generated new cluster ID: d1e16c31bfa74b5b900b07e0b7ea829c
I20260808 22:27:34.527194 20963 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:27:34.536839 20963 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:27:34.537932 20963 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:27:34.544227 20963 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44195cbc2abb441d89c021024447b35c: Generated new TSK 0
I20260808 22:27:34.544813 20963 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:27:34.557029 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.193:0
--local_ip_for_outbound_sockets=127.5.11.193
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:33871
--builtin_ntp_servers=127.5.11.212:39379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260808 22:27:34.670315 20980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:34.670538 20980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:34.670598 20980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:34.674638 20980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:34.674688 20980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:34.674769 20980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.193
I20260808 22:27:34.679282 20980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:39379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:33871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20980
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.193
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:34.680298 20980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:34.681610 20980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:34.690953 20993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:34.690964 20989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:34.690964 20988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:34.691092 20980 server_base.cc:1034] running on GCE node
I20260808 22:27:34.691594 20980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:34.692220 20980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:34.693383 20980 hybrid_clock.cc:648] HybridClock initialized: now 1786228054693368 us; error 26 us; skew 500 ppm
I20260808 22:27:34.695185 20980 webserver.cc:460] Webserver started at http://127.5.11.193:46847/ using document root <none> and password file <none>
I20260808 22:27:34.695797 20980 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:34.695878 20980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:34.696174 20980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:27:34.698172 20980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/data/instance:
uuid: "083e176bcfb249a79b5efb76c84fe946"
format_stamp: "Formatted at 2026-08-08 22:27:34 on dist-test-slave-7f01"
server_key: "5f1d617ef6b0ff4cf30b6a409e3b92db"
server_key_iv: "57ac3b0c82c1a7cd18d5f682a096a418"
server_key_version: "encryptionkey@0"
I20260808 22:27:34.698725 20980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance:
uuid: "083e176bcfb249a79b5efb76c84fe946"
format_stamp: "Formatted at 2026-08-08 22:27:34 on dist-test-slave-7f01"
server_key: "5f1d617ef6b0ff4cf30b6a409e3b92db"
server_key_iv: "57ac3b0c82c1a7cd18d5f682a096a418"
server_key_version: "encryptionkey@0"
I20260808 22:27:34.702378 20980 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 22:27:34.704807 21003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:34.705967 20980 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:27:34.706085 20980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/wal
uuid: "083e176bcfb249a79b5efb76c84fe946"
format_stamp: "Formatted at 2026-08-08 22:27:34 on dist-test-slave-7f01"
server_key: "5f1d617ef6b0ff4cf30b6a409e3b92db"
server_key_iv: "57ac3b0c82c1a7cd18d5f682a096a418"
server_key_version: "encryptionkey@0"
I20260808 22:27:34.706188 20980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:34.721896 20980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:34.722540 20980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:34.722728 20980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:34.723292 20980 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:27:34.724299 20980 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:27:34.724349 20980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:34.724416 20980 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:27:34.724455 20980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:34.733034 20980 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.193:37397
I20260808 22:27:34.733065 21174 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.193:37397 every 8 connection(s)
I20260808 22:27:34.733891 20980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
I20260808 22:27:34.734103 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 20980
I20260808 22:27:34.734210 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance
I20260808 22:27:34.734568 5167 external_mini_cluster.cc:1428] Setting key 75374b54dc9ad566d921406ab411b8f1
I20260808 22:27:34.736742 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.194:0
--local_ip_for_outbound_sockets=127.5.11.194
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:33871
--builtin_ntp_servers=127.5.11.212:39379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 22:27:34.745002 20946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:34.735826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:35913 (local address 127.5.11.254:33871)
0808 22:27:34.735978 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:34.735991 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:34.737299 (+ 1308us) server_negotiation.cc:407] Connection header received
0808 22:27:34.739243 (+ 1944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:34.739246 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:34.739303 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:34.739382 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:34.740472 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:34.741009 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:34.742885 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:34.743107 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:34.743189 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:34.744523 (+ 1334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:34.744526 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:34.744528 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:27:34.744540 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:34.744560 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:27:34.744568 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:34.744626 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:34.744824 (+ 198us) server_negotiation.cc:299] Negotiation successful
0808 22:27:34.744881 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 22:27:34.746219 21179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:34.736359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:33871 (local address 127.5.11.193:35913)
0808 22:27:34.737134 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:34.737190 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:34.739043 (+ 1853us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:34.739559 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:34.739565 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:34.739646 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:34.740299 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:34.740323 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:34.741143 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:34.741150 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:34.742755 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:34.742773 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:34.743276 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:34.743283 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:34.743527 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:34.744211 (+ 684us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:27:34.744214 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:34.744275 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:27:34.744365 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:34.744654 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:34.744666 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:34.744675 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 22:27:34.744812 (+ 137us) client_negotiation.cc:240] Negotiation successful
0808 22:27:34.745145 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":109,"threads_started":1}
I20260808 22:27:34.747476 21175 heartbeater.cc:347] Connected to a master server at 127.5.11.254:33871
I20260808 22:27:34.747705 21175 heartbeater.cc:464] Registering TS with master...
I20260808 22:27:34.748224 21175 heartbeater.cc:511] Master 127.5.11.254:33871 requested a full tablet report, sending...
I20260808 22:27:34.749698 20882 ts_manager.cc:194] Registered new tserver with Master: 083e176bcfb249a79b5efb76c84fe946 (127.5.11.193:37397)
I20260808 22:27:34.750918 20882 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.11.193:35913
W20260808 22:27:34.850708 21180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:34.850955 21180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:34.851020 21180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:34.855135 21180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:34.855209 21180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:34.855295 21180 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.194
I20260808 22:27:34.859922 21180 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:39379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:33871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21180
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.194
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:34.860951 21180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:34.862320 21180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:34.871582 21192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:34.871613 21190 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:34.871613 21189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:34.871884 21180 server_base.cc:1034] running on GCE node
I20260808 22:27:34.872309 21180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:34.872932 21180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:34.874118 21180 hybrid_clock.cc:648] HybridClock initialized: now 1786228054874104 us; error 33 us; skew 500 ppm
I20260808 22:27:34.875855 21180 webserver.cc:460] Webserver started at http://127.5.11.194:33811/ using document root <none> and password file <none>
I20260808 22:27:34.876492 21180 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:34.876580 21180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:34.876784 21180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:27:34.878782 21180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/data/instance:
uuid: "e1a4053a93c84c8394e10b7bbe840e81"
format_stamp: "Formatted at 2026-08-08 22:27:34 on dist-test-slave-7f01"
server_key: "e141df276f3d11f65e326cf3e2946d02"
server_key_iv: "3ac30b5faefbc59c84d6feb518f7f90e"
server_key_version: "encryptionkey@0"
I20260808 22:27:34.879349 21180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance:
uuid: "e1a4053a93c84c8394e10b7bbe840e81"
format_stamp: "Formatted at 2026-08-08 22:27:34 on dist-test-slave-7f01"
server_key: "e141df276f3d11f65e326cf3e2946d02"
server_key_iv: "3ac30b5faefbc59c84d6feb518f7f90e"
server_key_version: "encryptionkey@0"
I20260808 22:27:34.882932 21180 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260808 22:27:34.885350 21202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:34.886473 21180 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:27:34.886602 21180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/wal
uuid: "e1a4053a93c84c8394e10b7bbe840e81"
format_stamp: "Formatted at 2026-08-08 22:27:34 on dist-test-slave-7f01"
server_key: "e141df276f3d11f65e326cf3e2946d02"
server_key_iv: "3ac30b5faefbc59c84d6feb518f7f90e"
server_key_version: "encryptionkey@0"
I20260808 22:27:34.886710 21180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:34.899209 21180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:34.899854 21180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:34.900050 21180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:34.900602 21180 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:27:34.901535 21180 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:27:34.901607 21180 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:34.901677 21180 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:27:34.901718 21180 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:34.910671 21180 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.194:37987
I20260808 22:27:34.910753 21393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.194:37987 every 8 connection(s)
I20260808 22:27:34.911607 21180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
I20260808 22:27:34.914541 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 21180
I20260808 22:27:34.914659 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance
I20260808 22:27:34.914965 5167 external_mini_cluster.cc:1428] Setting key cb6bf50d45173bdc741846d9c8be4728
I20260808 22:27:34.917047 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.195:0
--local_ip_for_outbound_sockets=127.5.11.195
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:33871
--builtin_ntp_servers=127.5.11.212:39379
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260808 22:27:34.921043 20946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:34.913446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:37227 (local address 127.5.11.254:33871)
0808 22:27:34.913587 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:34.913591 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:34.914589 (+ 998us) server_negotiation.cc:407] Connection header received
0808 22:27:34.916221 (+ 1632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:34.916224 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:34.916280 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:34.916362 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:34.917325 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:34.917837 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:34.919076 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:34.919363 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:34.919482 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:34.920558 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:34.920561 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:34.920564 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:27:34.920582 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:34.920601 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:27:34.920608 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:34.920677 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:34.920841 (+ 164us) server_negotiation.cc:299] Negotiation successful
0808 22:27:34.920905 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 22:27:34.921866 21398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:34.913826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:33871 (local address 127.5.11.194:37227)
0808 22:27:34.914438 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:34.914479 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:34.916018 (+ 1539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:34.916526 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:34.916533 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:34.916616 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:34.917167 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:34.917192 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:34.917965 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:34.917970 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:34.918945 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:34.918957 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:34.919521 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:34.919525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:34.919700 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:34.920316 (+ 616us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:27:34.920318 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:34.920354 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:27:34.920414 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:34.920692 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:34.920700 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:34.920706 (+ 6us) client_negotiation.cc:769] Sending connection context
0808 22:27:34.920829 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 22:27:34.921104 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":104,"threads_started":1}
I20260808 22:27:34.922987 21394 heartbeater.cc:347] Connected to a master server at 127.5.11.254:33871
I20260808 22:27:34.923226 21394 heartbeater.cc:464] Registering TS with master...
I20260808 22:27:34.923722 21394 heartbeater.cc:511] Master 127.5.11.254:33871 requested a full tablet report, sending...
I20260808 22:27:34.924823 20882 ts_manager.cc:194] Registered new tserver with Master: e1a4053a93c84c8394e10b7bbe840e81 (127.5.11.194:37987)
I20260808 22:27:34.925680 20882 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.11.194:37227
W20260808 22:27:35.029848 21399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:35.030085 21399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:35.030120 21399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:35.033988 21399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:35.034037 21399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:35.034087 21399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.195
I20260808 22:27:35.038529 21399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:39379
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:33871
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.195
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:35.039427 21399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:35.040735 21399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:35.050359 21410 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:35.050350 21408 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:35.050670 21399 server_base.cc:1034] running on GCE node
W20260808 22:27:35.050354 21407 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:35.051059 21399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:35.051690 21399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:35.052896 21399 hybrid_clock.cc:648] HybridClock initialized: now 1786228055052879 us; error 29 us; skew 500 ppm
I20260808 22:27:35.054751 21399 webserver.cc:460] Webserver started at http://127.5.11.195:34243/ using document root <none> and password file <none>
I20260808 22:27:35.055330 21399 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:35.055393 21399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:35.055642 21399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:27:35.057680 21399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/data/instance:
uuid: "1be8a253117e4649aa2b5189cc513f64"
format_stamp: "Formatted at 2026-08-08 22:27:35 on dist-test-slave-7f01"
server_key: "6ab1af1cef412a96a278d1f5d3d5c8af"
server_key_iv: "93ec0d49d082535eb00a6fc2d38cd4b3"
server_key_version: "encryptionkey@0"
I20260808 22:27:35.058250 21399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance:
uuid: "1be8a253117e4649aa2b5189cc513f64"
format_stamp: "Formatted at 2026-08-08 22:27:35 on dist-test-slave-7f01"
server_key: "6ab1af1cef412a96a278d1f5d3d5c8af"
server_key_iv: "93ec0d49d082535eb00a6fc2d38cd4b3"
server_key_version: "encryptionkey@0"
I20260808 22:27:35.061954 21399 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260808 22:27:35.064492 21422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:35.065675 21399 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 22:27:35.065796 21399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/wal
uuid: "1be8a253117e4649aa2b5189cc513f64"
format_stamp: "Formatted at 2026-08-08 22:27:35 on dist-test-slave-7f01"
server_key: "6ab1af1cef412a96a278d1f5d3d5c8af"
server_key_iv: "93ec0d49d082535eb00a6fc2d38cd4b3"
server_key_version: "encryptionkey@0"
I20260808 22:27:35.065899 21399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:35.079066 21399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:35.079696 21399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:35.079936 21399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:35.080500 21399 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:27:35.081415 21399 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:27:35.081472 21399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:35.081537 21399 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:27:35.081570 21399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:35.090085 21399 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.195:40005
I20260808 22:27:35.090107 21607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.195:40005 every 8 connection(s)
I20260808 22:27:35.090962 21399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
I20260808 22:27:35.094292 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 21399
I20260808 22:27:35.094408 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance
I20260808 22:27:35.094714 5167 external_mini_cluster.cc:1428] Setting key 409b8536c56b00bc8852fbdff9ffe285
I20260808 22:27:35.100180 20946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:35.092769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:46045 (local address 127.5.11.254:33871)
0808 22:27:35.092895 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:35.092899 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:35.093897 (+ 998us) server_negotiation.cc:407] Connection header received
0808 22:27:35.095614 (+ 1717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:35.095618 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:35.095677 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:35.095808 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:35.096687 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:35.097219 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:35.098325 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:35.098549 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:35.098655 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:35.099673 (+ 1018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:35.099677 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:35.099680 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:27:35.099697 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:35.099717 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:27:35.099725 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:35.099821 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:35.099977 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 22:27:35.100037 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260808 22:27:35.100987 21611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:35.093119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:33871 (local address 127.5.11.195:46045)
0808 22:27:35.093754 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:35.093794 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:35.095392 (+ 1598us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:35.095982 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:35.095989 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:35.096070 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:35.096550 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:35.096563 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:35.097357 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:35.097361 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:35.098202 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:35.098211 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:35.098692 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:35.098696 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:35.098859 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:35.099410 (+ 551us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:27:35.099412 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:35.099458 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:27:35.099519 (+ 61us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:35.099834 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:35.099844 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:35.099855 (+ 11us) client_negotiation.cc:769] Sending connection context
0808 22:27:35.099974 (+ 119us) client_negotiation.cc:240] Negotiation successful
0808 22:27:35.100245 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":115,"threads_started":1}
I20260808 22:27:35.102001 21609 heartbeater.cc:347] Connected to a master server at 127.5.11.254:33871
I20260808 22:27:35.102231 21609 heartbeater.cc:464] Registering TS with master...
I20260808 22:27:35.102726 21609 heartbeater.cc:511] Master 127.5.11.254:33871 requested a full tablet report, sending...
I20260808 22:27:35.103691 20882 ts_manager.cc:194] Registered new tserver with Master: 1be8a253117e4649aa2b5189cc513f64 (127.5.11.195:40005)
I20260808 22:27:35.104287 20882 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.11.195:46045
I20260808 22:27:35.108500 5167 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 22:27:35.111246 20946 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:35.110394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41254 (local address 127.5.11.254:33871)
0808 22:27:35.110514 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:35.110519 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:35.110716 (+ 197us) server_negotiation.cc:407] Connection header received
0808 22:27:35.110808 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:35.110810 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:35.110846 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:35.110913 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:35.111111 (+ 198us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41254: BlockingRecv error: recv got EOF from 127.0.0.1:41254 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 22:27:35.111498 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 20980
I20260808 22:27:35.131225 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 21180
I20260808 22:27:35.147033 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 21399
I20260808 22:27:35.158972 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 20830
2026-08-08T22:27:35Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (872 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-08T22:27:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:27:35Z Disabled control of system clock
I20260808 22:27:35.193068 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:42155
--webserver_interface=127.5.11.254
--webserver_port=0
--builtin_ntp_servers=127.5.11.212:40331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:42155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 22:27:35.304304 21623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:35.304549 21623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:35.304586 21623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:35.308602 21623 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:27:35.308653 21623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:35.308669 21623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:35.308704 21623 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:27:35.308735 21623 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:27:35.313326 21623 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:40331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:42155
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:42155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:35.314330 21623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:35.315681 21623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:35.324048 21639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:35.324065 21633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:35.324065 21635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:35.324179 21623 server_base.cc:1034] running on GCE node
I20260808 22:27:35.324906 21623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:35.325956 21623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:35.327121 21623 hybrid_clock.cc:648] HybridClock initialized: now 1786228055327104 us; error 25 us; skew 500 ppm
I20260808 22:27:35.328958 21623 webserver.cc:460] Webserver started at http://127.5.11.254:41211/ using document root <none> and password file <none>
I20260808 22:27:35.329526 21623 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:35.329588 21623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:35.329833 21623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:27:35.331897 21623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "50a8b76e603e4587af3298daaa90bd4a"
format_stamp: "Formatted at 2026-08-08 22:27:35 on dist-test-slave-7f01"
server_key: "e1bc2676478a9a86e0e0ad156b7bd3d8"
server_key_iv: "133a5d52606f119a0f8d32464988bf88"
server_key_version: "encryptionkey@0"
I20260808 22:27:35.332458 21623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "50a8b76e603e4587af3298daaa90bd4a"
format_stamp: "Formatted at 2026-08-08 22:27:35 on dist-test-slave-7f01"
server_key: "e1bc2676478a9a86e0e0ad156b7bd3d8"
server_key_iv: "133a5d52606f119a0f8d32464988bf88"
server_key_version: "encryptionkey@0"
I20260808 22:27:35.336102 21623 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:27:35.338470 21647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:35.339677 21623 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:27:35.339848 21623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "50a8b76e603e4587af3298daaa90bd4a"
format_stamp: "Formatted at 2026-08-08 22:27:35 on dist-test-slave-7f01"
server_key: "e1bc2676478a9a86e0e0ad156b7bd3d8"
server_key_iv: "133a5d52606f119a0f8d32464988bf88"
server_key_version: "encryptionkey@0"
I20260808 22:27:35.339946 21623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:35.344420 21623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:35.345039 21623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:35.345212 21623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:35.351718 21623 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:42155
I20260808 22:27:35.351778 21727 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:42155 every 8 connection(s)
I20260808 22:27:35.352658 21623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:27:35.355858 21728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:27:35.358567 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 21623
I20260808 22:27:35.358702 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:27:35.359009 5167 external_mini_cluster.cc:1428] Setting key cb960c5c6da0b0accaca873f4151f9f2
I20260808 22:27:35.361883 21728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50a8b76e603e4587af3298daaa90bd4a: Bootstrap starting.
I20260808 22:27:35.364343 21728 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50a8b76e603e4587af3298daaa90bd4a: Neither blocks nor log segments found. Creating new log.
I20260808 22:27:35.364578 21734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:35.360472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60022 (local address 127.5.11.254:42155)
0808 22:27:35.361107 (+ 635us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:35.361122 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:35.361156 (+ 34us) server_negotiation.cc:407] Connection header received
0808 22:27:35.361753 (+ 597us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:35.361770 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:35.362153 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:35.362474 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:35.363269 (+ 795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:35.363274 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:35.363284 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:27:35.363294 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:35.363365 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:27:35.363382 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:35.363529 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:35.363601 (+ 72us) server_negotiation.cc:299] Negotiation successful
0808 22:27:35.363875 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":129,"threads_started":1}
I20260808 22:27:35.365245 21728 log.cc:824] T 00000000000000000000000000000000 P 50a8b76e603e4587af3298daaa90bd4a: Log is configured to *not* fsync() on all Append() calls
I20260808 22:27:35.367228 21728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50a8b76e603e4587af3298daaa90bd4a: No bootstrap required, opened a new log
I20260808 22:27:35.370532 21728 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50a8b76e603e4587af3298daaa90bd4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50a8b76e603e4587af3298daaa90bd4a" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 42155 } }
I20260808 22:27:35.370734 21728 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50a8b76e603e4587af3298daaa90bd4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:27:35.370775 21728 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50a8b76e603e4587af3298daaa90bd4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50a8b76e603e4587af3298daaa90bd4a, State: Initialized, Role: FOLLOWER
I20260808 22:27:35.371174 21728 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50a8b76e603e4587af3298daaa90bd4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50a8b76e603e4587af3298daaa90bd4a" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 42155 } }
I20260808 22:27:35.371331 21728 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50a8b76e603e4587af3298daaa90bd4a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:27:35.371413 21728 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50a8b76e603e4587af3298daaa90bd4a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:27:35.371527 21728 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50a8b76e603e4587af3298daaa90bd4a [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:27:35.372416 21728 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50a8b76e603e4587af3298daaa90bd4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50a8b76e603e4587af3298daaa90bd4a" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 42155 } }
I20260808 22:27:35.372738 21728 leader_election.cc:302] T 00000000000000000000000000000000 P 50a8b76e603e4587af3298daaa90bd4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50a8b76e603e4587af3298daaa90bd4a; no voters:
I20260808 22:27:35.373023 21728 leader_election.cc:288] T 00000000000000000000000000000000 P 50a8b76e603e4587af3298daaa90bd4a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:27:35.373102 21739 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50a8b76e603e4587af3298daaa90bd4a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:27:35.373346 21739 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50a8b76e603e4587af3298daaa90bd4a [term 1 LEADER]: Becoming Leader. State: Replica: 50a8b76e603e4587af3298daaa90bd4a, State: Running, Role: LEADER
I20260808 22:27:35.373673 21739 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50a8b76e603e4587af3298daaa90bd4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50a8b76e603e4587af3298daaa90bd4a" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 42155 } }
I20260808 22:27:35.374223 21728 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50a8b76e603e4587af3298daaa90bd4a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:27:35.375229 21741 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50a8b76e603e4587af3298daaa90bd4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50a8b76e603e4587af3298daaa90bd4a. Latest consensus state: current_term: 1 leader_uuid: "50a8b76e603e4587af3298daaa90bd4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50a8b76e603e4587af3298daaa90bd4a" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 42155 } } }
I20260808 22:27:35.375350 21741 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50a8b76e603e4587af3298daaa90bd4a [sys.catalog]: This master's current role is: LEADER
I20260808 22:27:35.375826 21740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50a8b76e603e4587af3298daaa90bd4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "50a8b76e603e4587af3298daaa90bd4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50a8b76e603e4587af3298daaa90bd4a" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 42155 } } }
I20260808 22:27:35.375972 21740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50a8b76e603e4587af3298daaa90bd4a [sys.catalog]: This master's current role is: LEADER
W20260808 22:27:35.378947 21765 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 50a8b76e603e4587af3298daaa90bd4a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:27:35.379029 21765 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:27:35.379268 21752 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:27:35.380198 21752 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:27:35.385350 21752 catalog_manager.cc:1347] Generated new cluster ID: 7617d3ad020c4f75aabbd82542a3c4d3
I20260808 22:27:35.385412 21752 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:27:35.400116 21752 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:27:35.400422 21752 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:27:35.409260 21752 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50a8b76e603e4587af3298daaa90bd4a: Generated new TSK 0
I20260808 22:27:35.410079 21752 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:27:35.420189 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.193:0
--local_ip_for_outbound_sockets=127.5.11.193
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:42155
--builtin_ntp_servers=127.5.11.212:40331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260808 22:27:35.534158 21768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:35.534437 21768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:35.534509 21768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:35.538417 21768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:35.538492 21768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:35.538578 21768 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.193
I20260808 22:27:35.543334 21768 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:40331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:42155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.193
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:35.544446 21768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:35.545804 21768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:35.555178 21781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:35.555204 21778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:35.555204 21779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:35.555361 21768 server_base.cc:1034] running on GCE node
I20260808 22:27:35.555774 21768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:35.556384 21768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:35.557569 21768 hybrid_clock.cc:648] HybridClock initialized: now 1786228055557553 us; error 27 us; skew 500 ppm
I20260808 22:27:35.559399 21768 webserver.cc:460] Webserver started at http://127.5.11.193:45031/ using document root <none> and password file <none>
I20260808 22:27:35.560026 21768 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:35.560115 21768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:35.560338 21768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:27:35.562331 21768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/data/instance:
uuid: "bec5f0bfc5ba40fa97168e8e74a819bf"
format_stamp: "Formatted at 2026-08-08 22:27:35 on dist-test-slave-7f01"
server_key: "7902917bd7af2ff377f3c4a8110cacfe"
server_key_iv: "700ad5ee67a59481981d78d89f02a5e4"
server_key_version: "encryptionkey@0"
I20260808 22:27:35.562886 21768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance:
uuid: "bec5f0bfc5ba40fa97168e8e74a819bf"
format_stamp: "Formatted at 2026-08-08 22:27:35 on dist-test-slave-7f01"
server_key: "7902917bd7af2ff377f3c4a8110cacfe"
server_key_iv: "700ad5ee67a59481981d78d89f02a5e4"
server_key_version: "encryptionkey@0"
I20260808 22:27:35.566507 21768 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260808 22:27:35.568948 21795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:35.570096 21768 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:27:35.570225 21768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/wal
uuid: "bec5f0bfc5ba40fa97168e8e74a819bf"
format_stamp: "Formatted at 2026-08-08 22:27:35 on dist-test-slave-7f01"
server_key: "7902917bd7af2ff377f3c4a8110cacfe"
server_key_iv: "700ad5ee67a59481981d78d89f02a5e4"
server_key_version: "encryptionkey@0"
I20260808 22:27:35.570330 21768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:35.574892 21768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:35.575516 21768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:35.575696 21768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:35.576248 21768 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:27:35.577144 21768 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:27:35.577215 21768 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:35.577279 21768 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:27:35.577324 21768 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:35.585876 21768 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.193:41339
I20260808 22:27:35.585904 21986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.193:41339 every 8 connection(s)
I20260808 22:27:35.586740 21768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
I20260808 22:27:35.587122 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 21768
I20260808 22:27:35.587277 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance
I20260808 22:27:35.587620 5167 external_mini_cluster.cc:1428] Setting key 5328bb51fd8505d95dd9ee823b2686d4
I20260808 22:27:35.589742 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.194:0
--local_ip_for_outbound_sockets=127.5.11.194
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:42155
--builtin_ntp_servers=127.5.11.212:40331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 22:27:35.593140 21734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:35.588566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:50299 (local address 127.5.11.254:42155)
0808 22:27:35.588697 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:35.588701 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:35.590000 (+ 1299us) server_negotiation.cc:407] Connection header received
0808 22:27:35.591324 (+ 1324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:35.591327 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:35.591370 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:35.591450 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:35.592704 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:35.592708 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:35.592710 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:27:35.592721 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:35.592742 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:27:35.592749 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:35.592828 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:35.592999 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 22:27:35.593035 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 22:27:35.594252 21990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:35.589092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:42155 (local address 127.5.11.193:50299)
0808 22:27:35.589851 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:35.589905 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:35.591122 (+ 1217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:35.591629 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:35.591641 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:35.591787 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:35.592378 (+ 591us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:27:35.592389 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:35.592449 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:27:35.592545 (+ 96us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:35.592835 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:35.592847 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:35.592856 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 22:27:35.593043 (+ 187us) client_negotiation.cc:240] Negotiation successful
0808 22:27:35.593288 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":162,"threads_started":1}
I20260808 22:27:35.595506 21987 heartbeater.cc:347] Connected to a master server at 127.5.11.254:42155
I20260808 22:27:35.595788 21987 heartbeater.cc:464] Registering TS with master...
I20260808 22:27:35.596309 21987 heartbeater.cc:511] Master 127.5.11.254:42155 requested a full tablet report, sending...
I20260808 22:27:35.598138 21669 ts_manager.cc:194] Registered new tserver with Master: bec5f0bfc5ba40fa97168e8e74a819bf (127.5.11.193:41339)
W20260808 22:27:35.704512 21993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:35.704746 21993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:35.704784 21993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:35.708694 21993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:35.708740 21993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:35.708792 21993 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.194
I20260808 22:27:35.713299 21993 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:40331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:42155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.194
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:35.714222 21993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:35.715473 21993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:35.724794 22004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:35.724781 22012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:35.724807 22006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:35.725299 21993 server_base.cc:1034] running on GCE node
I20260808 22:27:35.725710 21993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:35.726313 21993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:35.727470 21993 hybrid_clock.cc:648] HybridClock initialized: now 1786228055727460 us; error 27 us; skew 500 ppm
I20260808 22:27:35.729274 21993 webserver.cc:460] Webserver started at http://127.5.11.194:40217/ using document root <none> and password file <none>
I20260808 22:27:35.729851 21993 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:35.729912 21993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:35.730124 21993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:27:35.732184 21993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/data/instance:
uuid: "f24a927af8654a728a97b1dd589d97a5"
format_stamp: "Formatted at 2026-08-08 22:27:35 on dist-test-slave-7f01"
server_key: "3993e49058c172844b8e891f3e0b44ff"
server_key_iv: "24aa0f1a5e1455bfd755393dca284b2e"
server_key_version: "encryptionkey@0"
I20260808 22:27:35.732744 21993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance:
uuid: "f24a927af8654a728a97b1dd589d97a5"
format_stamp: "Formatted at 2026-08-08 22:27:35 on dist-test-slave-7f01"
server_key: "3993e49058c172844b8e891f3e0b44ff"
server_key_iv: "24aa0f1a5e1455bfd755393dca284b2e"
server_key_version: "encryptionkey@0"
I20260808 22:27:35.736291 21993 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 22:27:35.738696 22022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:35.739916 21993 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:27:35.740034 21993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/wal
uuid: "f24a927af8654a728a97b1dd589d97a5"
format_stamp: "Formatted at 2026-08-08 22:27:35 on dist-test-slave-7f01"
server_key: "3993e49058c172844b8e891f3e0b44ff"
server_key_iv: "24aa0f1a5e1455bfd755393dca284b2e"
server_key_version: "encryptionkey@0"
I20260808 22:27:35.740144 21993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:35.744776 21993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:35.745328 21993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:35.745512 21993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:35.746016 21993 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:27:35.746878 21993 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:27:35.746946 21993 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:35.747010 21993 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:27:35.747059 21993 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:35.755513 21993 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.194:46273
I20260808 22:27:35.755535 22200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.194:46273 every 8 connection(s)
I20260808 22:27:35.756477 21993 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
I20260808 22:27:35.757647 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 21993
I20260808 22:27:35.757792 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance
I20260808 22:27:35.758091 5167 external_mini_cluster.cc:1428] Setting key 13b9ceba72eb58ae61a4a33514216ed5
I20260808 22:27:35.760133 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.195:0
--local_ip_for_outbound_sockets=127.5.11.195
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:42155
--builtin_ntp_servers=127.5.11.212:40331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260808 22:27:35.762861 21734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:35.758266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:43047 (local address 127.5.11.254:42155)
0808 22:27:35.758432 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:35.758437 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:35.759595 (+ 1158us) server_negotiation.cc:407] Connection header received
0808 22:27:35.761069 (+ 1474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:35.761072 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:35.761114 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:35.761197 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:35.762418 (+ 1221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:35.762421 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:35.762423 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:27:35.762444 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:35.762466 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:27:35.762475 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:35.762547 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:35.762717 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 22:27:35.762755 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 22:27:35.763912 22205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:35.758741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:42155 (local address 127.5.11.194:43047)
0808 22:27:35.759453 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:35.759504 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:35.760861 (+ 1357us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:35.761366 (+ 505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:35.761374 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:35.761456 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:35.762084 (+ 628us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:27:35.762088 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:35.762146 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:27:35.762240 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:35.762556 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:35.762568 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:35.762576 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 22:27:35.762744 (+ 168us) client_negotiation.cc:240] Negotiation successful
0808 22:27:35.762983 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":138,"threads_started":1}
I20260808 22:27:35.764981 22201 heartbeater.cc:347] Connected to a master server at 127.5.11.254:42155
I20260808 22:27:35.765230 22201 heartbeater.cc:464] Registering TS with master...
I20260808 22:27:35.765714 22201 heartbeater.cc:511] Master 127.5.11.254:42155 requested a full tablet report, sending...
I20260808 22:27:35.766884 21669 ts_manager.cc:194] Registered new tserver with Master: f24a927af8654a728a97b1dd589d97a5 (127.5.11.194:46273)
W20260808 22:27:35.872926 22208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:35.873162 22208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:35.873199 22208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:35.877664 22208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:35.877732 22208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:35.877817 22208 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.195
I20260808 22:27:35.882942 22208 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:40331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:42155
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.195
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:35.884279 22208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:35.886058 22208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:35.896780 22217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:35.896764 22220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:35.896770 22218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:35.897150 22208 server_base.cc:1034] running on GCE node
I20260808 22:27:35.897562 22208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:35.898144 22208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:35.899300 22208 hybrid_clock.cc:648] HybridClock initialized: now 1786228055899255 us; error 58 us; skew 500 ppm
I20260808 22:27:35.901088 22208 webserver.cc:460] Webserver started at http://127.5.11.195:42297/ using document root <none> and password file <none>
I20260808 22:27:35.901670 22208 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:35.901732 22208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:35.901979 22208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:27:35.904068 22208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/data/instance:
uuid: "ba0b8a52e6834b889f65ad8b7f33877b"
format_stamp: "Formatted at 2026-08-08 22:27:35 on dist-test-slave-7f01"
server_key: "5d6f16626035e6871f4525ca029764af"
server_key_iv: "c79414b7d659a30fc31afb5d14622364"
server_key_version: "encryptionkey@0"
I20260808 22:27:35.904630 22208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance:
uuid: "ba0b8a52e6834b889f65ad8b7f33877b"
format_stamp: "Formatted at 2026-08-08 22:27:35 on dist-test-slave-7f01"
server_key: "5d6f16626035e6871f4525ca029764af"
server_key_iv: "c79414b7d659a30fc31afb5d14622364"
server_key_version: "encryptionkey@0"
I20260808 22:27:35.908193 22208 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:27:35.910647 22231 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:35.911921 22208 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:27:35.912060 22208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/wal
uuid: "ba0b8a52e6834b889f65ad8b7f33877b"
format_stamp: "Formatted at 2026-08-08 22:27:35 on dist-test-slave-7f01"
server_key: "5d6f16626035e6871f4525ca029764af"
server_key_iv: "c79414b7d659a30fc31afb5d14622364"
server_key_version: "encryptionkey@0"
I20260808 22:27:35.912168 22208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:35.916611 22208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:35.917186 22208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:35.917371 22208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:35.917894 22208 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:27:35.918776 22208 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:27:35.918846 22208 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:35.918910 22208 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:27:35.918956 22208 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:35.927848 22208 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.195:42973
I20260808 22:27:35.927853 22405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.195:42973 every 8 connection(s)
I20260808 22:27:35.928711 22208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
I20260808 22:27:35.929948 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 22208
I20260808 22:27:35.930034 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance
I20260808 22:27:35.930342 5167 external_mini_cluster.cc:1428] Setting key 77453c484a1fccad356f0fe028bd4e85
I20260808 22:27:35.934747 21734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:35.930462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:51937 (local address 127.5.11.254:42155)
0808 22:27:35.930608 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:35.930613 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:35.931824 (+ 1211us) server_negotiation.cc:407] Connection header received
0808 22:27:35.933262 (+ 1438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:35.933265 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:35.933300 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:35.933368 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:35.934369 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:35.934372 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:35.934374 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:27:35.934385 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:35.934407 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:27:35.934415 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:35.934486 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:35.934610 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 22:27:35.934646 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 22:27:35.935550 22408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:35.930857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:42155 (local address 127.5.11.195:51937)
0808 22:27:35.931665 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:35.931717 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:35.933070 (+ 1353us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:35.933522 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:35.933529 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:35.933611 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:35.934120 (+ 509us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:27:35.934122 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:35.934161 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:27:35.934225 (+ 64us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:35.934490 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:35.934496 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:35.934500 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:27:35.934623 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 22:27:35.934803 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":143,"threads_started":1}
I20260808 22:27:35.936638 22406 heartbeater.cc:347] Connected to a master server at 127.5.11.254:42155
I20260808 22:27:35.936862 22406 heartbeater.cc:464] Registering TS with master...
I20260808 22:27:35.937206 22406 heartbeater.cc:511] Master 127.5.11.254:42155 requested a full tablet report, sending...
I20260808 22:27:35.938089 21669 ts_manager.cc:194] Registered new tserver with Master: ba0b8a52e6834b889f65ad8b7f33877b (127.5.11.195:42973)
I20260808 22:27:35.944145 5167 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260808 22:27:35.946807 21734 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:27:35.946022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60024 (local address 127.5.11.254:42155)
0808 22:27:35.946161 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:35.946166 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:35.946302 (+ 136us) server_negotiation.cc:407] Connection header received
0808 22:27:35.946379 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:35.946381 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:35.946410 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:35.946479 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:35.946683 (+ 204us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60024: BlockingRecv error: recv got EOF from 127.0.0.1:60024 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 22:27:35.946921 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 21768
I20260808 22:27:35.961058 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 21993
I20260808 22:27:35.983639 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 22208
I20260808 22:27:35.996454 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 21623
2026-08-08T22:27:36Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (836 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:27:36 dist-test-slave-7f01 krb5kdc[22418](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:27:36 dist-test-slave-7f01 krb5kdc[22418](info): set up 2 sockets
Aug 08 22:27:36 dist-test-slave-7f01 krb5kdc[22418](info): commencing operation
krb5kdc: starting...
W20260808 22:27:37.253458 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:27:37 dist-test-slave-7f01 krb5kdc[22418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:27:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:27:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.11.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
I20260808 22:27:37.379144 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:33907
--webserver_interface=127.5.11.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:43523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:33907
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:27:37.490152 22438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:37.490412 22438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:37.490451 22438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:37.494400 22438 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:27:37.494450 22438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:37.494470 22438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:37.494493 22438 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:27:37.494508 22438 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:27:37.499706 22438 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:43523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:33907
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:33907
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:37.500774 22438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:37.501770 22438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:37.510473 22453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:37.510538 22438 server_base.cc:1034] running on GCE node
W20260808 22:27:37.510491 22448 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:37.510490 22449 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:37.511293 22438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:37.512410 22438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:37.513598 22438 hybrid_clock.cc:648] HybridClock initialized: now 1786228057513583 us; error 26 us; skew 500 ppm
Aug 08 22:27:37 dist-test-slave-7f01 krb5kdc[22418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228057, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:27:37.516366 22438 init.cc:377] Logged in from keytab as kudu/127.5.11.254@KRBTEST.COM (short username kudu)
I20260808 22:27:37.517210 22438 webserver.cc:460] Webserver started at http://127.5.11.254:37517/ using document root <none> and password file <none>
I20260808 22:27:37.517757 22438 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:37.517848 22438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:37.518100 22438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:27:37.520182 22438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "81447fecea9b4838b16e631668981fd1"
format_stamp: "Formatted at 2026-08-08 22:27:37 on dist-test-slave-7f01"
server_key: "d871a00bb3242d00419cf5e291434aa7"
server_key_iv: "8db996c047c6ed0d45fd44bc07fa78a8"
server_key_version: "encryptionkey@0"
I20260808 22:27:37.520752 22438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "81447fecea9b4838b16e631668981fd1"
format_stamp: "Formatted at 2026-08-08 22:27:37 on dist-test-slave-7f01"
server_key: "d871a00bb3242d00419cf5e291434aa7"
server_key_iv: "8db996c047c6ed0d45fd44bc07fa78a8"
server_key_version: "encryptionkey@0"
I20260808 22:27:37.524360 22438 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:27:37.526666 22471 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:37.527848 22438 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:27:37.527984 22438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "81447fecea9b4838b16e631668981fd1"
format_stamp: "Formatted at 2026-08-08 22:27:37 on dist-test-slave-7f01"
server_key: "d871a00bb3242d00419cf5e291434aa7"
server_key_iv: "8db996c047c6ed0d45fd44bc07fa78a8"
server_key_version: "encryptionkey@0"
I20260808 22:27:37.528103 22438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:37.544356 22438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:37.547224 22438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:37.547427 22438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:37.554235 22438 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:33907
I20260808 22:27:37.554522 22544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:33907 every 8 connection(s)
I20260808 22:27:37.555212 22438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:27:37.557983 22545 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:27:37.563494 22545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81447fecea9b4838b16e631668981fd1: Bootstrap starting.
I20260808 22:27:37.565433 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 22438
I20260808 22:27:37.565554 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:27:37.565851 5167 external_mini_cluster.cc:1428] Setting key f25b8a21990e072a6bb6dfc8bb69608d
I20260808 22:27:37.566231 22545 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81447fecea9b4838b16e631668981fd1: Neither blocks nor log segments found. Creating new log.
I20260808 22:27:37.567209 22545 log.cc:824] T 00000000000000000000000000000000 P 81447fecea9b4838b16e631668981fd1: Log is configured to *not* fsync() on all Append() calls
I20260808 22:27:37.569329 22545 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81447fecea9b4838b16e631668981fd1: No bootstrap required, opened a new log
I20260808 22:27:37.573423 22545 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81447fecea9b4838b16e631668981fd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81447fecea9b4838b16e631668981fd1" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 33907 } }
I20260808 22:27:37.573732 22545 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81447fecea9b4838b16e631668981fd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:27:37.573804 22545 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81447fecea9b4838b16e631668981fd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81447fecea9b4838b16e631668981fd1, State: Initialized, Role: FOLLOWER
I20260808 22:27:37.574321 22545 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81447fecea9b4838b16e631668981fd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81447fecea9b4838b16e631668981fd1" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 33907 } }
I20260808 22:27:37.574455 22545 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81447fecea9b4838b16e631668981fd1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:27:37.574548 22545 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81447fecea9b4838b16e631668981fd1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:27:37.574647 22545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81447fecea9b4838b16e631668981fd1 [term 0 FOLLOWER]: Advancing to term 1
Aug 08 22:27:37 dist-test-slave-7f01 krb5kdc[22418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:27:37.575654 22545 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81447fecea9b4838b16e631668981fd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81447fecea9b4838b16e631668981fd1" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 33907 } }
I20260808 22:27:37.576067 22545 leader_election.cc:302] T 00000000000000000000000000000000 P 81447fecea9b4838b16e631668981fd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81447fecea9b4838b16e631668981fd1; no voters:
I20260808 22:27:37.576509 22545 leader_election.cc:288] T 00000000000000000000000000000000 P 81447fecea9b4838b16e631668981fd1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:27:37.576695 22554 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81447fecea9b4838b16e631668981fd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:27:37.576922 22554 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81447fecea9b4838b16e631668981fd1 [term 1 LEADER]: Becoming Leader. State: Replica: 81447fecea9b4838b16e631668981fd1, State: Running, Role: LEADER
I20260808 22:27:37.577275 22554 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81447fecea9b4838b16e631668981fd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81447fecea9b4838b16e631668981fd1" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 33907 } }
I20260808 22:27:37.577865 22545 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81447fecea9b4838b16e631668981fd1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:27:37.578536 22556 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81447fecea9b4838b16e631668981fd1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81447fecea9b4838b16e631668981fd1. Latest consensus state: current_term: 1 leader_uuid: "81447fecea9b4838b16e631668981fd1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81447fecea9b4838b16e631668981fd1" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 33907 } } }
I20260808 22:27:37.579330 22556 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81447fecea9b4838b16e631668981fd1 [sys.catalog]: This master's current role is: LEADER
I20260808 22:27:37.579217 22558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81447fecea9b4838b16e631668981fd1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81447fecea9b4838b16e631668981fd1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81447fecea9b4838b16e631668981fd1" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 33907 } } }
I20260808 22:27:37.579663 22558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81447fecea9b4838b16e631668981fd1 [sys.catalog]: This master's current role is: LEADER
I20260808 22:27:37.581419 22552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:37.567072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54544 (local address 127.5.11.254:33907)
0808 22:27:37.567482 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:37.567499 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:37.567536 (+ 37us) server_negotiation.cc:407] Connection header received
0808 22:27:37.568358 (+ 822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:37.568380 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:37.568698 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:37.568978 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:37.569866 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:37.570801 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:37.571968 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:37.572458 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:37.572573 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:37.575436 (+ 2863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:37.575443 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:37.575462 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:37.575501 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:37.577548 (+ 2047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:37.578101 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:37.578107 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:37.578113 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:37.578258 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:37.578596 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:37.578603 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:37.578607 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:37.579197 (+ 590us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:37.579395 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:37.580161 (+ 766us) server_negotiation.cc:299] Negotiation successful
0808 22:27:37.580493 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":127,"threads_started":1}
W20260808 22:27:37.583256 22574 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 81447fecea9b4838b16e631668981fd1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:27:37.583348 22574 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:27:37.583458 22569 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:27:37.584291 22569 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:27:37.589146 22569 catalog_manager.cc:1347] Generated new cluster ID: ae13d06a845a4ca2a643016b401fd639
I20260808 22:27:37.589262 22569 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:27:37.602912 22569 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:27:37.603910 22569 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:27:37.609333 22569 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81447fecea9b4838b16e631668981fd1: Generated new TSK 0
I20260808 22:27:37.609977 22569 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.11.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab.
Entry for principal kudu/127.5.11.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab.
I20260808 22:27:37.646667 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.193:0
--local_ip_for_outbound_sockets=127.5.11.193
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:33907
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:43523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:27:37.760237 22586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:37.760474 22586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:37.760515 22586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:37.764456 22586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:37.764505 22586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:37.764559 22586 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.193
I20260808 22:27:37.769496 22586 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:43523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:33907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22586
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.193
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:37.770385 22586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:37.771312 22586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:37.781090 22598 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:37.781106 22596 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:37.781092 22593 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:37.781296 22586 server_base.cc:1034] running on GCE node
I20260808 22:27:37.781733 22586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:37.782353 22586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:37.783542 22586 hybrid_clock.cc:648] HybridClock initialized: now 1786228057783525 us; error 26 us; skew 500 ppm
Aug 08 22:27:37 dist-test-slave-7f01 krb5kdc[22418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228057, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:27:37.786319 22586 init.cc:377] Logged in from keytab as kudu/127.5.11.193@KRBTEST.COM (short username kudu)
I20260808 22:27:37.787165 22586 webserver.cc:460] Webserver started at http://127.5.11.193:37871/ using document root <none> and password file <none>
I20260808 22:27:37.787674 22586 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:37.787775 22586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:37.787976 22586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:27:37.789921 22586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/data/instance:
uuid: "b843a852eeef40ef8d4d14f0bf6ffdc8"
format_stamp: "Formatted at 2026-08-08 22:27:37 on dist-test-slave-7f01"
server_key: "f84b61eaf72929f0f2068ae666d70050"
server_key_iv: "ba02b35b2389b82f390f082ff0620534"
server_key_version: "encryptionkey@0"
I20260808 22:27:37.790448 22586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance:
uuid: "b843a852eeef40ef8d4d14f0bf6ffdc8"
format_stamp: "Formatted at 2026-08-08 22:27:37 on dist-test-slave-7f01"
server_key: "f84b61eaf72929f0f2068ae666d70050"
server_key_iv: "ba02b35b2389b82f390f082ff0620534"
server_key_version: "encryptionkey@0"
I20260808 22:27:37.794126 22586 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.003s
I20260808 22:27:37.796427 22609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:37.797608 22586 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 22:27:37.797730 22586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/wal
uuid: "b843a852eeef40ef8d4d14f0bf6ffdc8"
format_stamp: "Formatted at 2026-08-08 22:27:37 on dist-test-slave-7f01"
server_key: "f84b61eaf72929f0f2068ae666d70050"
server_key_iv: "ba02b35b2389b82f390f082ff0620534"
server_key_version: "encryptionkey@0"
I20260808 22:27:37.797827 22586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:37.820276 22586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:37.823524 22586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:37.823675 22586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:37.824304 22586 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:27:37.825124 22586 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:27:37.825172 22586 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:37.825240 22586 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:27:37.825270 22586 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:37.834347 22586 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.193:46087
I20260808 22:27:37.834378 22782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.193:46087 every 8 connection(s)
I20260808 22:27:37.835212 22586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
I20260808 22:27:37.843400 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 22586
I20260808 22:27:37.843549 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance
I20260808 22:27:37.843889 5167 external_mini_cluster.cc:1428] Setting key d2614bc0dd0303dad82ca0cc4cfd2a7a
Aug 08 22:27:37 dist-test-slave-7f01 krb5kdc[22418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228057, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:27:37.849539 22552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:37.836936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:40875 (local address 127.5.11.254:33907)
0808 22:27:37.837083 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:37.837086 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:37.837863 (+ 777us) server_negotiation.cc:407] Connection header received
0808 22:27:37.839359 (+ 1496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:37.839362 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:37.839424 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:37.839545 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:37.840768 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:37.841333 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:37.842505 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:37.842743 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:37.842851 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:37.845707 (+ 2856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:37.845714 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:37.845718 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:37.845748 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:37.847435 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:37.847999 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:37.848004 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:37.848006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:37.848111 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:37.848469 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:37.848472 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:37.848474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:37.848657 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:37.848762 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:37.849244 (+ 482us) server_negotiation.cc:299] Negotiation successful
0808 22:27:37.849367 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 22:27:37.850363 22788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:37.837298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:33907 (local address 127.5.11.193:40875)
0808 22:27:37.837718 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:37.837760 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:37.839031 (+ 1271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:37.839718 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:37.839725 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:37.840147 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:37.840619 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:37.840632 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:37.841463 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:37.841467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:37.842377 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:37.842387 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:37.842904 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:37.842908 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:37.843072 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:37.843719 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:37.843769 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:37.845543 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:37.847547 (+ 2004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:37.847552 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:37.847562 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:37.847884 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:37.848213 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:37.848217 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:37.848219 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:37.848364 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:37.848761 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:37.848768 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:37.849010 (+ 242us) client_negotiation.cc:769] Sending connection context
0808 22:27:37.849187 (+ 177us) client_negotiation.cc:240] Negotiation successful
0808 22:27:37.849508 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":109,"threads_started":1}
I20260808 22:27:37.851714 22785 heartbeater.cc:347] Connected to a master server at 127.5.11.254:33907
I20260808 22:27:37.851985 22785 heartbeater.cc:464] Registering TS with master...
I20260808 22:27:37.852459 22785 heartbeater.cc:511] Master 127.5.11.254:33907 requested a full tablet report, sending...
I20260808 22:27:37.854163 22496 ts_manager.cc:194] Registered new tserver with Master: b843a852eeef40ef8d4d14f0bf6ffdc8 (127.5.11.193:46087)
WARNING: no policy specified for kudu/127.5.11.194@KRBTEST.COM; defaulting to no policy
I20260808 22:27:37.855830 22496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.193@KRBTEST.COM'} at 127.5.11.193:40875
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab.
Entry for principal kudu/127.5.11.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab.
I20260808 22:27:37.874302 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.194:0
--local_ip_for_outbound_sockets=127.5.11.194
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:33907
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:43523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:27:37.988642 22793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:37.988922 22793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:37.988998 22793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:37.992987 22793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:37.993059 22793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:37.993141 22793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.194
I20260808 22:27:37.998234 22793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:43523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:33907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.194
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:37.999259 22793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:38.000268 22793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:38.009984 22802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:38.009943 22804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:38.009950 22810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:38.010306 22793 server_base.cc:1034] running on GCE node
I20260808 22:27:38.010707 22793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:38.011288 22793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:38.012481 22793 hybrid_clock.cc:648] HybridClock initialized: now 1786228058012465 us; error 24 us; skew 500 ppm
Aug 08 22:27:38 dist-test-slave-7f01 krb5kdc[22418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228058, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:27:38.018153 22793 init.cc:377] Logged in from keytab as kudu/127.5.11.194@KRBTEST.COM (short username kudu)
I20260808 22:27:38.018987 22793 webserver.cc:460] Webserver started at http://127.5.11.194:41635/ using document root <none> and password file <none>
I20260808 22:27:38.019547 22793 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:38.019611 22793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:38.019980 22793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:27:38.022033 22793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/data/instance:
uuid: "8803cc9d8dce4c00b018bd891acec664"
format_stamp: "Formatted at 2026-08-08 22:27:38 on dist-test-slave-7f01"
server_key: "905bd546739e698ff3aec7fcce482d4b"
server_key_iv: "b33149dbaac295cc2e43eb1b86131c9d"
server_key_version: "encryptionkey@0"
I20260808 22:27:38.022598 22793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance:
uuid: "8803cc9d8dce4c00b018bd891acec664"
format_stamp: "Formatted at 2026-08-08 22:27:38 on dist-test-slave-7f01"
server_key: "905bd546739e698ff3aec7fcce482d4b"
server_key_iv: "b33149dbaac295cc2e43eb1b86131c9d"
server_key_version: "encryptionkey@0"
I20260808 22:27:38.026311 22793 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 22:27:38.028699 22820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:38.029850 22793 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:27:38.029968 22793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/wal
uuid: "8803cc9d8dce4c00b018bd891acec664"
format_stamp: "Formatted at 2026-08-08 22:27:38 on dist-test-slave-7f01"
server_key: "905bd546739e698ff3aec7fcce482d4b"
server_key_iv: "b33149dbaac295cc2e43eb1b86131c9d"
server_key_version: "encryptionkey@0"
I20260808 22:27:38.030062 22793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:38.047909 22793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:38.050833 22793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:38.051038 22793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:38.051631 22793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:27:38.052488 22793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:27:38.052536 22793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:38.052600 22793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:27:38.052637 22793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:38.061487 22793 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.194:42137
I20260808 22:27:38.061522 23001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.194:42137 every 8 connection(s)
I20260808 22:27:38.062397 22793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
I20260808 22:27:38.071149 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 22793
I20260808 22:27:38.071266 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance
Aug 08 22:27:38 dist-test-slave-7f01 krb5kdc[22418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228058, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:27:38.071578 5167 external_mini_cluster.cc:1428] Setting key ba71ff6c59b443a5d984edd6e4620761
I20260808 22:27:38.075589 22552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:38.064147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:36587 (local address 127.5.11.254:33907)
0808 22:27:38.064314 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:38.064319 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:38.065117 (+ 798us) server_negotiation.cc:407] Connection header received
0808 22:27:38.066396 (+ 1279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:38.066398 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:38.066445 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:38.066519 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:38.067635 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.068323 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.069385 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.069593 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.069668 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:38.072121 (+ 2453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:38.072125 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:38.072127 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:38.072157 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:38.073599 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:38.074095 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:38.074100 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:38.074101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:38.074184 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:38.074531 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:38.074536 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:38.074538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:38.074716 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:38.074820 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:38.075284 (+ 464us) server_negotiation.cc:299] Negotiation successful
0808 22:27:38.075408 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 22:27:38.076447 23004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:38.064527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:33907 (local address 127.5.11.194:36587)
0808 22:27:38.064962 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:38.065013 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:38.066210 (+ 1197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:38.066681 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:38.066687 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:38.067047 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:38.067502 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:38.067514 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.068426 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.068429 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:38.069280 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:38.069289 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.069714 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.069717 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:38.069852 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:38.070371 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:38.070386 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:38.071959 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:38.073717 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:38.073723 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:38.073736 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:38.073994 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:38.074273 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:38.074277 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:38.074279 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:38.074432 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:38.074820 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:38.074827 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:38.075056 (+ 229us) client_negotiation.cc:769] Sending connection context
0808 22:27:38.075239 (+ 183us) client_negotiation.cc:240] Negotiation successful
0808 22:27:38.075543 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":122,"threads_started":1}
I20260808 22:27:38.077652 23002 heartbeater.cc:347] Connected to a master server at 127.5.11.254:33907
I20260808 22:27:38.077881 23002 heartbeater.cc:464] Registering TS with master...
I20260808 22:27:38.078373 23002 heartbeater.cc:511] Master 127.5.11.254:33907 requested a full tablet report, sending...
I20260808 22:27:38.079435 22496 ts_manager.cc:194] Registered new tserver with Master: 8803cc9d8dce4c00b018bd891acec664 (127.5.11.194:42137)
I20260808 22:27:38.080163 22496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.194@KRBTEST.COM'} at 127.5.11.194:36587
WARNING: no policy specified for kudu/127.5.11.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab.
Entry for principal kudu/127.5.11.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab.
I20260808 22:27:38.101207 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.195:0
--local_ip_for_outbound_sockets=127.5.11.195
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:33907
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:43523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:27:38.213451 23007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:27:38.213694 23007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:27:38.213734 23007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:27:38.217630 23007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:27:38.217685 23007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:27:38.217741 23007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.195
I20260808 22:27:38.222882 23007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:43523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:33907
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.195
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:27:38.223865 23007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:27:38.224819 23007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:27:38.234323 23012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:27:38.234346 23013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:38.234575 23007 server_base.cc:1034] running on GCE node
W20260808 22:27:38.234330 23015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:27:38.234993 23007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:27:38.235625 23007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:27:38.236815 23007 hybrid_clock.cc:648] HybridClock initialized: now 1786228058236779 us; error 42 us; skew 500 ppm
Aug 08 22:27:38 dist-test-slave-7f01 krb5kdc[22418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228058, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:27:38.239524 23007 init.cc:377] Logged in from keytab as kudu/127.5.11.195@KRBTEST.COM (short username kudu)
I20260808 22:27:38.240384 23007 webserver.cc:460] Webserver started at http://127.5.11.195:45839/ using document root <none> and password file <none>
I20260808 22:27:38.240918 23007 fs_manager.cc:359] Metadata directory not provided
I20260808 22:27:38.241006 23007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:27:38.241226 23007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:27:38.243202 23007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/data/instance:
uuid: "9f3ec4815d344512b4fbd3c2c12c52ca"
format_stamp: "Formatted at 2026-08-08 22:27:38 on dist-test-slave-7f01"
server_key: "900a87573be83ae5328bb8019aae61db"
server_key_iv: "1743f784b07349b2088b918473ba258d"
server_key_version: "encryptionkey@0"
I20260808 22:27:38.243805 23007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance:
uuid: "9f3ec4815d344512b4fbd3c2c12c52ca"
format_stamp: "Formatted at 2026-08-08 22:27:38 on dist-test-slave-7f01"
server_key: "900a87573be83ae5328bb8019aae61db"
server_key_iv: "1743f784b07349b2088b918473ba258d"
server_key_version: "encryptionkey@0"
I20260808 22:27:38.247293 23007 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.003s
I20260808 22:27:38.249642 23027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:38.250792 23007 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:27:38.250921 23007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/wal
uuid: "9f3ec4815d344512b4fbd3c2c12c52ca"
format_stamp: "Formatted at 2026-08-08 22:27:38 on dist-test-slave-7f01"
server_key: "900a87573be83ae5328bb8019aae61db"
server_key_iv: "1743f784b07349b2088b918473ba258d"
server_key_version: "encryptionkey@0"
I20260808 22:27:38.251025 23007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:27:38.275130 23007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:27:38.278427 23007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:27:38.278621 23007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:27:38.279289 23007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:27:38.280218 23007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:27:38.280282 23007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:38.280352 23007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:27:38.280400 23007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:27:38.289113 23007 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.195:44273
I20260808 22:27:38.289147 23215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.195:44273 every 8 connection(s)
I20260808 22:27:38.290019 23007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
I20260808 22:27:38.297950 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 23007
I20260808 22:27:38.298096 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance
I20260808 22:27:38.298394 5167 external_mini_cluster.cc:1428] Setting key ba20ad7d11c210cf18a1922bb0844bf1
Aug 08 22:27:38 dist-test-slave-7f01 krb5kdc[22418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228058, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:27:38.303385 22552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:38.291802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:58477 (local address 127.5.11.254:33907)
0808 22:27:38.291940 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:38.291944 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:38.292728 (+ 784us) server_negotiation.cc:407] Connection header received
0808 22:27:38.294111 (+ 1383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:38.294114 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:38.294172 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:38.294256 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:38.295408 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.296082 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.297200 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.297432 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.297545 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:38.300267 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:38.300271 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:38.300273 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:38.300302 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:38.301679 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:38.302129 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:38.302131 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:38.302133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:38.302209 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:38.302500 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:38.302507 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:38.302509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:38.302679 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:38.302756 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:38.303179 (+ 423us) server_negotiation.cc:299] Negotiation successful
0808 22:27:38.303268 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 22:27:38.304229 23220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:38.292152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:33907 (local address 127.5.11.195:58477)
0808 22:27:38.292584 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:38.292624 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:38.293893 (+ 1269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:38.294418 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:38.294425 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:38.294768 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:27:38.295257 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:38.295271 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.296202 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.296206 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:38.297071 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:38.297080 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.297591 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.297595 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:38.297765 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:38.298335 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:27:38.298351 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:27:38.300116 (+ 1765us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:27:38.301784 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:38.301789 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:38.301800 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:38.302046 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:38.302285 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:27:38.302287 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:27:38.302288 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:27:38.302414 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:27:38.302764 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:27:38.302769 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:27:38.302980 (+ 211us) client_negotiation.cc:769] Sending connection context
0808 22:27:38.303129 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 22:27:38.303356 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":110,"threads_started":1}
I20260808 22:27:38.305204 23216 heartbeater.cc:347] Connected to a master server at 127.5.11.254:33907
I20260808 22:27:38.305428 23216 heartbeater.cc:464] Registering TS with master...
I20260808 22:27:38.305879 23216 heartbeater.cc:511] Master 127.5.11.254:33907 requested a full tablet report, sending...
I20260808 22:27:38.306818 22496 ts_manager.cc:194] Registered new tserver with Master: 9f3ec4815d344512b4fbd3c2c12c52ca (127.5.11.195:44273)
I20260808 22:27:38.307364 22496 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.195@KRBTEST.COM'} at 127.5.11.195:58477
I20260808 22:27:38.311897 5167 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:27:38.320395 22552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:38.313806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54560 (local address 127.5.11.254:33907)
0808 22:27:38.313946 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:38.313950 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:38.314081 (+ 131us) server_negotiation.cc:407] Connection header received
0808 22:27:38.314159 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:38.314161 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:38.314196 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:38.314257 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:27:38.314837 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.315308 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.316158 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.316360 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.316436 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:38.317672 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:27:38.317674 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:27:38.317676 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:27:38.317693 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:27:38.318889 (+ 1196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:38.319251 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:38.319253 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:38.319254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:38.319325 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:27:38.319548 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:27:38.319550 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:27:38.319551 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:27:38.319785 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:27:38.319892 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:38.320124 (+ 232us) server_negotiation.cc:299] Negotiation successful
0808 22:27:38.320210 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 22:27:38.323011 22496 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54560:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:27:38.326033 22496 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:27:38.339929 23229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:38.334061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:46087 (local address 127.0.0.1:42066)
0808 22:27:38.334549 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:38.334595 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:38.334723 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:38.335499 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:38.335508 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:38.335535 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:27:38.335672 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:38.335680 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.337064 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.337070 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:38.338274 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:38.338284 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.339350 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.339353 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:38.339466 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:38.339514 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 22:27:38.339632 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 22:27:38.339774 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":84,"threads_started":1}
I20260808 22:27:38.340137 23231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:38.334259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42066 (local address 127.5.11.193:46087)
0808 22:27:38.334677 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:38.334697 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:38.334722 (+ 25us) server_negotiation.cc:407] Connection header received
0808 22:27:38.334935 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:38.334943 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:38.335175 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:38.335325 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:27:38.335924 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.336940 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.338415 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.339223 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.339333 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:38.339492 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:38.339881 (+ 389us) server_negotiation.cc:299] Negotiation successful
0808 22:27:38.339987 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":84,"threads_started":1}
I20260808 22:27:38.340178 23230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:38.334764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:44273 (local address 127.0.0.1:57580)
0808 22:27:38.335621 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:38.335636 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:38.335703 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:38.336421 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:38.336424 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:38.336444 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:27:38.336534 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:38.336539 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.337793 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.337799 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:38.338803 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:38.338812 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.339671 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.339678 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:38.339876 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:38.339900 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 22:27:38.339972 (+ 72us) client_negotiation.cc:240] Negotiation successful
0808 22:27:38.340055 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":778,"mutex_wait_us":40,"thread_start_us":93,"threads_started":1}
I20260808 22:27:38.340297 23232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:38.334167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:42137 (local address 127.0.0.1:55426)
0808 22:27:38.335004 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:38.335035 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:38.335175 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:38.335664 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:38.335671 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:38.335691 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:27:38.335843 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:38.335849 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.337548 (+ 1699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.337551 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:38.338662 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:38.338671 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.339773 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.339778 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:38.339926 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:38.339950 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 22:27:38.340025 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 22:27:38.340091 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":773,"mutex_wait_us":24,"thread_start_us":98,"threads_started":1}
I20260808 22:27:38.340541 23234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:38.335241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57580 (local address 127.5.11.195:44273)
0808 22:27:38.335977 (+ 736us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:38.335988 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:38.336009 (+ 21us) server_negotiation.cc:407] Connection header received
0808 22:27:38.336080 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:38.336085 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:38.336257 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:38.336411 (+ 154us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:27:38.336651 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.337678 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.338917 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.339545 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.339652 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:38.339801 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:38.340289 (+ 488us) server_negotiation.cc:299] Negotiation successful
0808 22:27:38.340388 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":103,"threads_started":1}
I20260808 22:27:38.341033 23233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:38.334830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55426 (local address 127.5.11.194:42137)
0808 22:27:38.335199 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:38.335213 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:38.335241 (+ 28us) server_negotiation.cc:407] Connection header received
0808 22:27:38.335318 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:38.335323 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:38.335504 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:38.336003 (+ 499us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:27:38.336156 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.337408 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.338795 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.339619 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.340593 (+ 974us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:38.340748 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:38.340799 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 22:27:38.340888 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":156,"threads_started":1}
I20260808 22:27:38.343293 23103 tablet_service.cc:1467] Processing CreateTablet for tablet 2b828b6c21ed422ba68418dabc3407a6 (DEFAULT_TABLE table=test-table [id=638d03af6cc1444a917f985c8fea8427]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:27:38.343292 22679 tablet_service.cc:1467] Processing CreateTablet for tablet 2b828b6c21ed422ba68418dabc3407a6 (DEFAULT_TABLE table=test-table [id=638d03af6cc1444a917f985c8fea8427]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:27:38.344247 22679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b828b6c21ed422ba68418dabc3407a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:27:38.344256 23103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b828b6c21ed422ba68418dabc3407a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:27:38.347309 22895 tablet_service.cc:1467] Processing CreateTablet for tablet 2b828b6c21ed422ba68418dabc3407a6 (DEFAULT_TABLE table=test-table [id=638d03af6cc1444a917f985c8fea8427]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:27:38.348316 22895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2b828b6c21ed422ba68418dabc3407a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:27:38.350836 23236 tablet_bootstrap.cc:492] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca: Bootstrap starting.
I20260808 22:27:38.351534 23237 tablet_bootstrap.cc:492] T 2b828b6c21ed422ba68418dabc3407a6 P b843a852eeef40ef8d4d14f0bf6ffdc8: Bootstrap starting.
I20260808 22:27:38.353291 23236 tablet_bootstrap.cc:654] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca: Neither blocks nor log segments found. Creating new log.
I20260808 22:27:38.353627 23237 tablet_bootstrap.cc:654] T 2b828b6c21ed422ba68418dabc3407a6 P b843a852eeef40ef8d4d14f0bf6ffdc8: Neither blocks nor log segments found. Creating new log.
I20260808 22:27:38.354084 23236 log.cc:824] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca: Log is configured to *not* fsync() on all Append() calls
I20260808 22:27:38.354477 23237 log.cc:824] T 2b828b6c21ed422ba68418dabc3407a6 P b843a852eeef40ef8d4d14f0bf6ffdc8: Log is configured to *not* fsync() on all Append() calls
I20260808 22:27:38.355451 23240 tablet_bootstrap.cc:492] T 2b828b6c21ed422ba68418dabc3407a6 P 8803cc9d8dce4c00b018bd891acec664: Bootstrap starting.
I20260808 22:27:38.356446 23237 tablet_bootstrap.cc:492] T 2b828b6c21ed422ba68418dabc3407a6 P b843a852eeef40ef8d4d14f0bf6ffdc8: No bootstrap required, opened a new log
I20260808 22:27:38.356643 23237 ts_tablet_manager.cc:1397] T 2b828b6c21ed422ba68418dabc3407a6 P b843a852eeef40ef8d4d14f0bf6ffdc8: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260808 22:27:38.356667 23236 tablet_bootstrap.cc:492] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca: No bootstrap required, opened a new log
I20260808 22:27:38.356822 23236 ts_tablet_manager.cc:1397] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260808 22:27:38.357817 23240 tablet_bootstrap.cc:654] T 2b828b6c21ed422ba68418dabc3407a6 P 8803cc9d8dce4c00b018bd891acec664: Neither blocks nor log segments found. Creating new log.
I20260808 22:27:38.358461 23240 log.cc:824] T 2b828b6c21ed422ba68418dabc3407a6 P 8803cc9d8dce4c00b018bd891acec664: Log is configured to *not* fsync() on all Append() calls
I20260808 22:27:38.360039 23240 tablet_bootstrap.cc:492] T 2b828b6c21ed422ba68418dabc3407a6 P 8803cc9d8dce4c00b018bd891acec664: No bootstrap required, opened a new log
I20260808 22:27:38.360242 23240 ts_tablet_manager.cc:1397] T 2b828b6c21ed422ba68418dabc3407a6 P 8803cc9d8dce4c00b018bd891acec664: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 22:27:38.360558 23236 raft_consensus.cc:348] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b843a852eeef40ef8d4d14f0bf6ffdc8" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46087 } } peers { permanent_uuid: "8803cc9d8dce4c00b018bd891acec664" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 42137 } } peers { permanent_uuid: "9f3ec4815d344512b4fbd3c2c12c52ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 44273 } }
I20260808 22:27:38.360859 23236 raft_consensus.cc:374] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:27:38.360939 23236 raft_consensus.cc:729] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f3ec4815d344512b4fbd3c2c12c52ca, State: Initialized, Role: FOLLOWER
I20260808 22:27:38.360795 23237 raft_consensus.cc:348] T 2b828b6c21ed422ba68418dabc3407a6 P b843a852eeef40ef8d4d14f0bf6ffdc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b843a852eeef40ef8d4d14f0bf6ffdc8" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46087 } } peers { permanent_uuid: "8803cc9d8dce4c00b018bd891acec664" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 42137 } } peers { permanent_uuid: "9f3ec4815d344512b4fbd3c2c12c52ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 44273 } }
I20260808 22:27:38.361073 23237 raft_consensus.cc:374] T 2b828b6c21ed422ba68418dabc3407a6 P b843a852eeef40ef8d4d14f0bf6ffdc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:27:38.361131 23237 raft_consensus.cc:729] T 2b828b6c21ed422ba68418dabc3407a6 P b843a852eeef40ef8d4d14f0bf6ffdc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b843a852eeef40ef8d4d14f0bf6ffdc8, State: Initialized, Role: FOLLOWER
I20260808 22:27:38.361407 23236 consensus_queue.cc:260] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b843a852eeef40ef8d4d14f0bf6ffdc8" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46087 } } peers { permanent_uuid: "8803cc9d8dce4c00b018bd891acec664" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 42137 } } peers { permanent_uuid: "9f3ec4815d344512b4fbd3c2c12c52ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 44273 } }
I20260808 22:27:38.361510 23237 consensus_queue.cc:260] T 2b828b6c21ed422ba68418dabc3407a6 P b843a852eeef40ef8d4d14f0bf6ffdc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b843a852eeef40ef8d4d14f0bf6ffdc8" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46087 } } peers { permanent_uuid: "8803cc9d8dce4c00b018bd891acec664" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 42137 } } peers { permanent_uuid: "9f3ec4815d344512b4fbd3c2c12c52ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 44273 } }
I20260808 22:27:38.362094 23216 heartbeater.cc:503] Master 127.5.11.254:33907 was elected leader, sending a full tablet report...
I20260808 22:27:38.362421 23237 ts_tablet_manager.cc:1428] T 2b828b6c21ed422ba68418dabc3407a6 P b843a852eeef40ef8d4d14f0bf6ffdc8: Time spent starting tablet: real 0.006s user 0.000s sys 0.005s
I20260808 22:27:38.362421 23236 ts_tablet_manager.cc:1428] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260808 22:27:38.363291 22785 heartbeater.cc:503] Master 127.5.11.254:33907 was elected leader, sending a full tablet report...
I20260808 22:27:38.364270 23240 raft_consensus.cc:348] T 2b828b6c21ed422ba68418dabc3407a6 P 8803cc9d8dce4c00b018bd891acec664 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b843a852eeef40ef8d4d14f0bf6ffdc8" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46087 } } peers { permanent_uuid: "8803cc9d8dce4c00b018bd891acec664" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 42137 } } peers { permanent_uuid: "9f3ec4815d344512b4fbd3c2c12c52ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 44273 } }
I20260808 22:27:38.366204 23240 raft_consensus.cc:374] T 2b828b6c21ed422ba68418dabc3407a6 P 8803cc9d8dce4c00b018bd891acec664 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:27:38.366277 23240 raft_consensus.cc:729] T 2b828b6c21ed422ba68418dabc3407a6 P 8803cc9d8dce4c00b018bd891acec664 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8803cc9d8dce4c00b018bd891acec664, State: Initialized, Role: FOLLOWER
I20260808 22:27:38.366777 23240 consensus_queue.cc:260] T 2b828b6c21ed422ba68418dabc3407a6 P 8803cc9d8dce4c00b018bd891acec664 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b843a852eeef40ef8d4d14f0bf6ffdc8" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46087 } } peers { permanent_uuid: "8803cc9d8dce4c00b018bd891acec664" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 42137 } } peers { permanent_uuid: "9f3ec4815d344512b4fbd3c2c12c52ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 44273 } }
I20260808 22:27:38.367847 23240 ts_tablet_manager.cc:1428] T 2b828b6c21ed422ba68418dabc3407a6 P 8803cc9d8dce4c00b018bd891acec664: Time spent starting tablet: real 0.008s user 0.006s sys 0.000s
I20260808 22:27:38.367980 23002 heartbeater.cc:503] Master 127.5.11.254:33907 was elected leader, sending a full tablet report...
W20260808 22:27:38.540951 23218 tablet.cc:2369] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 22:27:38.563525 23003 tablet.cc:2369] T 2b828b6c21ed422ba68418dabc3407a6 P 8803cc9d8dce4c00b018bd891acec664: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:27:38.572077 23243 raft_consensus.cc:482] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:27:38.572291 23243 raft_consensus.cc:504] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b843a852eeef40ef8d4d14f0bf6ffdc8" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46087 } } peers { permanent_uuid: "8803cc9d8dce4c00b018bd891acec664" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 42137 } } peers { permanent_uuid: "9f3ec4815d344512b4fbd3c2c12c52ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 44273 } }
I20260808 22:27:38.573385 23243 leader_election.cc:288] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b843a852eeef40ef8d4d14f0bf6ffdc8 (127.5.11.193:46087), 8803cc9d8dce4c00b018bd891acec664 (127.5.11.194:42137)
I20260808 22:27:38.574218 23249 raft_consensus.cc:482] T 2b828b6c21ed422ba68418dabc3407a6 P 8803cc9d8dce4c00b018bd891acec664 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:27:38.574376 23249 raft_consensus.cc:504] T 2b828b6c21ed422ba68418dabc3407a6 P 8803cc9d8dce4c00b018bd891acec664 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b843a852eeef40ef8d4d14f0bf6ffdc8" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46087 } } peers { permanent_uuid: "8803cc9d8dce4c00b018bd891acec664" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 42137 } } peers { permanent_uuid: "9f3ec4815d344512b4fbd3c2c12c52ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 44273 } }
I20260808 22:27:38.575457 23249 leader_election.cc:288] T 2b828b6c21ed422ba68418dabc3407a6 P 8803cc9d8dce4c00b018bd891acec664 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b843a852eeef40ef8d4d14f0bf6ffdc8 (127.5.11.193:46087), 9f3ec4815d344512b4fbd3c2c12c52ca (127.5.11.195:44273)
I20260808 22:27:38.577812 23220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:38.573592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:46087 (local address 127.5.11.195:39301)
0808 22:27:38.573767 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:38.573782 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:38.573872 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:38.574228 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:38.574233 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:38.574258 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:27:38.574380 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:38.574389 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.575234 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.575240 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:38.576432 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:38.576443 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.577411 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.577415 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:38.577537 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:38.577560 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 22:27:38.577628 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 22:27:38.577691 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98}
I20260808 22:27:38.578145 23233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:38.573757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:35053 (local address 127.5.11.194:42137)
0808 22:27:38.573956 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:38.573962 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:38.574120 (+ 158us) server_negotiation.cc:407] Connection header received
0808 22:27:38.574267 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:38.574273 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:38.574331 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:38.574380 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:27:38.574772 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.575458 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.576880 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.577490 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.577573 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:38.577616 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:38.577958 (+ 342us) server_negotiation.cc:299] Negotiation successful
0808 22:27:38.578021 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260808 22:27:38.578298 23231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:38.573680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:39301 (local address 127.5.11.193:46087)
0808 22:27:38.573796 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:38.573804 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:38.573823 (+ 19us) server_negotiation.cc:407] Connection header received
0808 22:27:38.574022 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:38.574026 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:38.574110 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:38.574185 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:27:38.574507 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.575102 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.576585 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.577193 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.577299 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:38.577353 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:38.578109 (+ 756us) server_negotiation.cc:299] Negotiation successful
0808 22:27:38.578180 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 22:27:38.578559 23251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:38.573611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:42137 (local address 127.5.11.195:35053)
0808 22:27:38.574007 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:38.574022 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:38.574132 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:38.574541 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:38.574543 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:38.574561 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:27:38.574659 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:38.574664 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.575606 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.575610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:38.576752 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:38.576759 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.577708 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.577713 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:38.577820 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:38.577846 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 22:27:38.578284 (+ 438us) client_negotiation.cc:240] Negotiation successful
0808 22:27:38.578394 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"mutex_wait_us":3,"thread_start_us":113,"threads_started":1}
I20260808 22:27:38.578917 22709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b828b6c21ed422ba68418dabc3407a6" candidate_uuid: "9f3ec4815d344512b4fbd3c2c12c52ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b843a852eeef40ef8d4d14f0bf6ffdc8" is_pre_election: true
I20260808 22:27:38.579221 22709 raft_consensus.cc:2454] T 2b828b6c21ed422ba68418dabc3407a6 P b843a852eeef40ef8d4d14f0bf6ffdc8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f3ec4815d344512b4fbd3c2c12c52ca in term 0.
I20260808 22:27:38.579576 22921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b828b6c21ed422ba68418dabc3407a6" candidate_uuid: "9f3ec4815d344512b4fbd3c2c12c52ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8803cc9d8dce4c00b018bd891acec664" is_pre_election: true
I20260808 22:27:38.579728 23034 leader_election.cc:302] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f3ec4815d344512b4fbd3c2c12c52ca, b843a852eeef40ef8d4d14f0bf6ffdc8; no voters:
I20260808 22:27:38.579797 22921 raft_consensus.cc:2454] T 2b828b6c21ed422ba68418dabc3407a6 P 8803cc9d8dce4c00b018bd891acec664 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f3ec4815d344512b4fbd3c2c12c52ca in term 0.
I20260808 22:27:38.580044 23243 raft_consensus.cc:2781] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:27:38.580140 23243 raft_consensus.cc:482] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:27:38.580199 23243 raft_consensus.cc:3037] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:27:38.580864 23234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:38.576360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:42905 (local address 127.5.11.195:44273)
0808 22:27:38.576453 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:38.576457 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:38.576468 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:27:38.576505 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:38.576508 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:38.576541 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:38.576602 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:27:38.576939 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.577617 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.579697 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.580288 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.580408 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:38.580450 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:38.580695 (+ 245us) server_negotiation.cc:299] Negotiation successful
0808 22:27:38.580761 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 22:27:38.580892 23253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:38.575897 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:44273 (local address 127.5.11.194:42905)
0808 22:27:38.576194 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:38.576207 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:38.576290 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:38.576693 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:38.576696 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:38.576712 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:27:38.576826 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:38.576835 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.577766 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.577769 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:38.579560 (+ 1791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:38.579571 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.580395 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.580401 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:38.580564 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:38.580588 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 22:27:38.580663 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 22:27:38.580724 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"mutex_wait_us":31,"thread_start_us":76,"threads_started":1}
I20260808 22:27:38.580904 23004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:38.575638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:46087 (local address 127.5.11.194:37135)
0808 22:27:38.576099 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:38.576112 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:27:38.576185 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:27:38.576803 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:27:38.576806 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:27:38.576823 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:27:38.576917 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:38.576922 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.577969 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.577973 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:38.579085 (+ 1112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:27:38.579096 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.580422 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.580426 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:27:38.580607 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:38.580628 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 22:27:38.580704 (+ 76us) client_negotiation.cc:240] Negotiation successful
0808 22:27:38.580774 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"mutex_wait_us":46}
I20260808 22:27:38.581192 23252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:38.575730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:37135 (local address 127.5.11.193:46087)
0808 22:27:38.576559 (+ 829us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:38.576570 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:38.576584 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:27:38.576617 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:38.576619 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:38.576670 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:38.577022 (+ 352us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:27:38.577120 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.577837 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.579212 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.580288 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.580581 (+ 293us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:38.580625 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:38.580953 (+ 328us) server_negotiation.cc:299] Negotiation successful
0808 22:27:38.581040 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":721,"thread_start_us":99,"threads_started":1}
I20260808 22:27:38.581499 23138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b828b6c21ed422ba68418dabc3407a6" candidate_uuid: "8803cc9d8dce4c00b018bd891acec664" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f3ec4815d344512b4fbd3c2c12c52ca" is_pre_election: true
I20260808 22:27:38.581699 23243 raft_consensus.cc:504] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b843a852eeef40ef8d4d14f0bf6ffdc8" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46087 } } peers { permanent_uuid: "8803cc9d8dce4c00b018bd891acec664" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 42137 } } peers { permanent_uuid: "9f3ec4815d344512b4fbd3c2c12c52ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 44273 } }
I20260808 22:27:38.581758 22709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b828b6c21ed422ba68418dabc3407a6" candidate_uuid: "8803cc9d8dce4c00b018bd891acec664" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b843a852eeef40ef8d4d14f0bf6ffdc8" is_pre_election: true
I20260808 22:27:38.581890 22709 raft_consensus.cc:2454] T 2b828b6c21ed422ba68418dabc3407a6 P b843a852eeef40ef8d4d14f0bf6ffdc8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8803cc9d8dce4c00b018bd891acec664 in term 0.
I20260808 22:27:38.581962 23138 raft_consensus.cc:2379] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 8803cc9d8dce4c00b018bd891acec664 in current term 1: Already voted for candidate 9f3ec4815d344512b4fbd3c2c12c52ca in this term.
I20260808 22:27:38.582078 23243 leader_election.cc:288] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca [CANDIDATE]: Term 1 election: Requested vote from peers b843a852eeef40ef8d4d14f0bf6ffdc8 (127.5.11.193:46087), 8803cc9d8dce4c00b018bd891acec664 (127.5.11.194:42137)
I20260808 22:27:38.582267 22829 leader_election.cc:302] T 2b828b6c21ed422ba68418dabc3407a6 P 8803cc9d8dce4c00b018bd891acec664 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8803cc9d8dce4c00b018bd891acec664, b843a852eeef40ef8d4d14f0bf6ffdc8; no voters:
I20260808 22:27:38.582509 22921 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b828b6c21ed422ba68418dabc3407a6" candidate_uuid: "9f3ec4815d344512b4fbd3c2c12c52ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8803cc9d8dce4c00b018bd891acec664"
I20260808 22:27:38.582540 23249 raft_consensus.cc:2781] T 2b828b6c21ed422ba68418dabc3407a6 P 8803cc9d8dce4c00b018bd891acec664 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:27:38.582643 22921 raft_consensus.cc:3037] T 2b828b6c21ed422ba68418dabc3407a6 P 8803cc9d8dce4c00b018bd891acec664 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:27:38.583079 22709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b828b6c21ed422ba68418dabc3407a6" candidate_uuid: "9f3ec4815d344512b4fbd3c2c12c52ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b843a852eeef40ef8d4d14f0bf6ffdc8"
I20260808 22:27:38.583197 22709 raft_consensus.cc:3037] T 2b828b6c21ed422ba68418dabc3407a6 P b843a852eeef40ef8d4d14f0bf6ffdc8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:27:38.583912 22921 raft_consensus.cc:2454] T 2b828b6c21ed422ba68418dabc3407a6 P 8803cc9d8dce4c00b018bd891acec664 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f3ec4815d344512b4fbd3c2c12c52ca in term 1.
I20260808 22:27:38.584026 23249 raft_consensus.cc:482] T 2b828b6c21ed422ba68418dabc3407a6 P 8803cc9d8dce4c00b018bd891acec664 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:27:38.584115 23249 raft_consensus.cc:3037] T 2b828b6c21ed422ba68418dabc3407a6 P 8803cc9d8dce4c00b018bd891acec664 [term 1 FOLLOWER]: Advancing to term 2
I20260808 22:27:38.584268 23029 leader_election.cc:302] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8803cc9d8dce4c00b018bd891acec664, 9f3ec4815d344512b4fbd3c2c12c52ca; no voters:
I20260808 22:27:38.584442 23243 raft_consensus.cc:2781] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:27:38.584566 22709 raft_consensus.cc:2454] T 2b828b6c21ed422ba68418dabc3407a6 P b843a852eeef40ef8d4d14f0bf6ffdc8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f3ec4815d344512b4fbd3c2c12c52ca in term 1.
I20260808 22:27:38.584635 23243 raft_consensus.cc:686] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca [term 1 LEADER]: Becoming Leader. State: Replica: 9f3ec4815d344512b4fbd3c2c12c52ca, State: Running, Role: LEADER
I20260808 22:27:38.584964 23243 consensus_queue.cc:237] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b843a852eeef40ef8d4d14f0bf6ffdc8" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46087 } } peers { permanent_uuid: "8803cc9d8dce4c00b018bd891acec664" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 42137 } } peers { permanent_uuid: "9f3ec4815d344512b4fbd3c2c12c52ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 44273 } }
I20260808 22:27:38.585212 23249 raft_consensus.cc:504] T 2b828b6c21ed422ba68418dabc3407a6 P 8803cc9d8dce4c00b018bd891acec664 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b843a852eeef40ef8d4d14f0bf6ffdc8" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46087 } } peers { permanent_uuid: "8803cc9d8dce4c00b018bd891acec664" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 42137 } } peers { permanent_uuid: "9f3ec4815d344512b4fbd3c2c12c52ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 44273 } }
I20260808 22:27:38.585906 23249 leader_election.cc:288] T 2b828b6c21ed422ba68418dabc3407a6 P 8803cc9d8dce4c00b018bd891acec664 [CANDIDATE]: Term 2 election: Requested vote from peers b843a852eeef40ef8d4d14f0bf6ffdc8 (127.5.11.193:46087), 9f3ec4815d344512b4fbd3c2c12c52ca (127.5.11.195:44273)
I20260808 22:27:38.586218 22709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b828b6c21ed422ba68418dabc3407a6" candidate_uuid: "8803cc9d8dce4c00b018bd891acec664" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b843a852eeef40ef8d4d14f0bf6ffdc8"
I20260808 22:27:38.586288 23138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2b828b6c21ed422ba68418dabc3407a6" candidate_uuid: "8803cc9d8dce4c00b018bd891acec664" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f3ec4815d344512b4fbd3c2c12c52ca"
I20260808 22:27:38.586334 22709 raft_consensus.cc:3037] T 2b828b6c21ed422ba68418dabc3407a6 P b843a852eeef40ef8d4d14f0bf6ffdc8 [term 1 FOLLOWER]: Advancing to term 2
W20260808 22:27:38.586524 22786 tablet.cc:2369] T 2b828b6c21ed422ba68418dabc3407a6 P b843a852eeef40ef8d4d14f0bf6ffdc8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:27:38.587170 22709 raft_consensus.cc:2454] T 2b828b6c21ed422ba68418dabc3407a6 P b843a852eeef40ef8d4d14f0bf6ffdc8 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8803cc9d8dce4c00b018bd891acec664 in term 2.
I20260808 22:27:38.587450 22829 leader_election.cc:302] T 2b828b6c21ed422ba68418dabc3407a6 P 8803cc9d8dce4c00b018bd891acec664 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 8803cc9d8dce4c00b018bd891acec664, b843a852eeef40ef8d4d14f0bf6ffdc8; no voters: 9f3ec4815d344512b4fbd3c2c12c52ca
I20260808 22:27:38.587666 23249 raft_consensus.cc:2781] T 2b828b6c21ed422ba68418dabc3407a6 P 8803cc9d8dce4c00b018bd891acec664 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 22:27:38.587904 23249 raft_consensus.cc:686] T 2b828b6c21ed422ba68418dabc3407a6 P 8803cc9d8dce4c00b018bd891acec664 [term 2 LEADER]: Becoming Leader. State: Replica: 8803cc9d8dce4c00b018bd891acec664, State: Running, Role: LEADER
I20260808 22:27:38.588225 23249 consensus_queue.cc:237] T 2b828b6c21ed422ba68418dabc3407a6 P 8803cc9d8dce4c00b018bd891acec664 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b843a852eeef40ef8d4d14f0bf6ffdc8" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46087 } } peers { permanent_uuid: "8803cc9d8dce4c00b018bd891acec664" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 42137 } } peers { permanent_uuid: "9f3ec4815d344512b4fbd3c2c12c52ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 44273 } }
I20260808 22:27:38.589090 22495 catalog_manager.cc:5495] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f3ec4815d344512b4fbd3c2c12c52ca (127.5.11.195). New cstate: current_term: 1 leader_uuid: "9f3ec4815d344512b4fbd3c2c12c52ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b843a852eeef40ef8d4d14f0bf6ffdc8" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8803cc9d8dce4c00b018bd891acec664" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 42137 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f3ec4815d344512b4fbd3c2c12c52ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 44273 } health_report { overall_health: HEALTHY } } }
I20260808 22:27:38.591578 22496 catalog_manager.cc:5495] T 2b828b6c21ed422ba68418dabc3407a6 P 8803cc9d8dce4c00b018bd891acec664 reported cstate change: term changed from 1 to 2, leader changed from 9f3ec4815d344512b4fbd3c2c12c52ca (127.5.11.195) to 8803cc9d8dce4c00b018bd891acec664 (127.5.11.194). New cstate: current_term: 2 leader_uuid: "8803cc9d8dce4c00b018bd891acec664" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b843a852eeef40ef8d4d14f0bf6ffdc8" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46087 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8803cc9d8dce4c00b018bd891acec664" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 42137 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9f3ec4815d344512b4fbd3c2c12c52ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 44273 } health_report { overall_health: UNKNOWN } } }
I20260808 22:27:38.630167 23233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:38.626950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55428 (local address 127.5.11.194:42137)
0808 22:27:38.627062 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 22:27:38.627066 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:27:38.627079 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:27:38.627202 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:27:38.627204 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:27:38.627256 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:27:38.627339 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:27:38.627535 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.628040 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.628874 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:27:38.629083 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:27:38.629161 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:27:38.629400 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:27:38.629794 (+ 394us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:27:38.629882 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 22:27:38.629933 (+ 51us) server_negotiation.cc:299] Negotiation successful
0808 22:27:38.629975 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 22:27:38.635908 23138 raft_consensus.cc:3032] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca [term 1 LEADER]: Stepping down as leader of term 1
I20260808 22:27:38.636013 23138 raft_consensus.cc:729] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 9f3ec4815d344512b4fbd3c2c12c52ca, State: Running, Role: LEADER
I20260808 22:27:38.636112 22709 raft_consensus.cc:1261] T 2b828b6c21ed422ba68418dabc3407a6 P b843a852eeef40ef8d4d14f0bf6ffdc8 [term 2 FOLLOWER]: Refusing update from remote peer 8803cc9d8dce4c00b018bd891acec664: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260808 22:27:38.636198 23138 consensus_queue.cc:260] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b843a852eeef40ef8d4d14f0bf6ffdc8" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46087 } } peers { permanent_uuid: "8803cc9d8dce4c00b018bd891acec664" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 42137 } } peers { permanent_uuid: "9f3ec4815d344512b4fbd3c2c12c52ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 44273 } }
I20260808 22:27:38.636442 23138 raft_consensus.cc:3037] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca [term 1 FOLLOWER]: Advancing to term 2
I20260808 22:27:38.636751 23249 consensus_queue.cc:1035] T 2b828b6c21ed422ba68418dabc3407a6 P 8803cc9d8dce4c00b018bd891acec664 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b843a852eeef40ef8d4d14f0bf6ffdc8" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:27:38.637774 23138 raft_consensus.cc:1261] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca [term 2 FOLLOWER]: Refusing update from remote peer 8803cc9d8dce4c00b018bd891acec664: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch)
I20260808 22:27:38.638278 23249 consensus_queue.cc:1035] T 2b828b6c21ed422ba68418dabc3407a6 P 8803cc9d8dce4c00b018bd891acec664 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f3ec4815d344512b4fbd3c2c12c52ca" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 44273 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:27:38.638396 23249 consensus_queue.cc:1225] T 2b828b6c21ed422ba68418dabc3407a6 P 8803cc9d8dce4c00b018bd891acec664 [LEADER]: Peer 9f3ec4815d344512b4fbd3c2c12c52ca log is divergent from this leader: its last log entry 1.1 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 0
I20260808 22:27:38.639303 23138 pending_rounds.cc:76] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca: Aborting all ops after (but not including) 0
I20260808 22:27:38.639374 23138 pending_rounds.cc:97] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca: Aborting uncommitted NO_OP operation due to leader change: 1.1
I20260808 22:27:38.639459 23138 raft_consensus.cc:2866] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca [term 2 FOLLOWER]: NO_OP replication failed: Aborted: Op aborted by new leader
I20260808 22:27:38.647580 23266 mvcc.cc:205] Tried to move back new op lower bound from 7316390128165863424 to 7316390127979859968. Current Snapshot: MvccSnapshot[applied={T|T < 7316390128165863424}]
I20260808 22:27:38.651319 22495 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54560:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:27:38.651487 22495 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54560:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:27:38.653875 22495 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 81447fecea9b4838b16e631668981fd1: Sending DeleteTablet for 3 replicas of tablet 2b828b6c21ed422ba68418dabc3407a6
I20260808 22:27:38.654583 22895 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b828b6c21ed422ba68418dabc3407a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:27:38 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:55426
I20260808 22:27:38.654652 22679 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b828b6c21ed422ba68418dabc3407a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:27:38 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:42066
I20260808 22:27:38.654850 23103 tablet_service.cc:1514] Processing DeleteTablet for tablet 2b828b6c21ed422ba68418dabc3407a6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:27:38 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:57580
I20260808 22:27:38.655236 23280 tablet_replica.cc:330] T 2b828b6c21ed422ba68418dabc3407a6 P b843a852eeef40ef8d4d14f0bf6ffdc8: stopping tablet replica
I20260808 22:27:38.655407 23279 tablet_replica.cc:330] T 2b828b6c21ed422ba68418dabc3407a6 P 8803cc9d8dce4c00b018bd891acec664: stopping tablet replica
I20260808 22:27:38.655541 23280 raft_consensus.cc:2229] T 2b828b6c21ed422ba68418dabc3407a6 P b843a852eeef40ef8d4d14f0bf6ffdc8 [term 2 FOLLOWER]: Raft consensus shutting down.
I20260808 22:27:38.655825 23279 raft_consensus.cc:2229] T 2b828b6c21ed422ba68418dabc3407a6 P 8803cc9d8dce4c00b018bd891acec664 [term 2 LEADER]: Raft consensus shutting down.
I20260808 22:27:38.655856 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 22586
I20260808 22:27:38.655853 23280 raft_consensus.cc:2258] T 2b828b6c21ed422ba68418dabc3407a6 P b843a852eeef40ef8d4d14f0bf6ffdc8 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260808 22:27:38.659332 23279 raft_consensus.cc:2258] T 2b828b6c21ed422ba68418dabc3407a6 P 8803cc9d8dce4c00b018bd891acec664 [term 2 FOLLOWER]: Raft consensus is shut down!
I20260808 22:27:38.660339 23284 tablet_replica.cc:330] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca: stopping tablet replica
I20260808 22:27:38.660676 23284 raft_consensus.cc:2229] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca [term 2 FOLLOWER]: Raft consensus shutting down.
I20260808 22:27:38.660935 23284 raft_consensus.cc:2258] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca [term 2 FOLLOWER]: Raft consensus is shut down!
I20260808 22:27:38.661103 23279 ts_tablet_manager.cc:1905] T 2b828b6c21ed422ba68418dabc3407a6 P 8803cc9d8dce4c00b018bd891acec664: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:27:38.662510 23284 ts_tablet_manager.cc:1905] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:27:38.664759 23279 ts_tablet_manager.cc:1918] T 2b828b6c21ed422ba68418dabc3407a6 P 8803cc9d8dce4c00b018bd891acec664: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260808 22:27:38.664862 23279 log.cc:1196] T 2b828b6c21ed422ba68418dabc3407a6 P 8803cc9d8dce4c00b018bd891acec664: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-1/wal/wals/2b828b6c21ed422ba68418dabc3407a6
I20260808 22:27:38.665179 23279 ts_tablet_manager.cc:1939] T 2b828b6c21ed422ba68418dabc3407a6 P 8803cc9d8dce4c00b018bd891acec664: Deleting consensus metadata
I20260808 22:27:38.665361 23284 ts_tablet_manager.cc:1918] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 2.2
I20260808 22:27:38.665514 23284 log.cc:1196] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786227991988852-5167-0/minicluster-data/ts-2/wal/wals/2b828b6c21ed422ba68418dabc3407a6
I20260808 22:27:38.665903 23284 ts_tablet_manager.cc:1939] T 2b828b6c21ed422ba68418dabc3407a6 P 9f3ec4815d344512b4fbd3c2c12c52ca: Deleting consensus metadata
I20260808 22:27:38.666342 22474 catalog_manager.cc:4841] TS 8803cc9d8dce4c00b018bd891acec664 (127.5.11.194:42137): tablet 2b828b6c21ed422ba68418dabc3407a6 (table test-table [id=638d03af6cc1444a917f985c8fea8427]) successfully deleted
I20260808 22:27:38.667037 22476 catalog_manager.cc:4841] TS 9f3ec4815d344512b4fbd3c2c12c52ca (127.5.11.195:44273): tablet 2b828b6c21ed422ba68418dabc3407a6 (table test-table [id=638d03af6cc1444a917f985c8fea8427]) successfully deleted
W20260808 22:27:38.667274 22476 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.11.193:46087 (error 108)
I20260808 22:27:38.667572 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 22793
I20260808 22:27:38.670902 23229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:27:38.670549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:46087 (local address 127.0.0.1:42076)
0808 22:27:38.670715 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 22:27:38.670799 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.193:46087: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260808 22:27:38.671074 22476 catalog_manager.cc:4584] TS b843a852eeef40ef8d4d14f0bf6ffdc8 (127.5.11.193:46087): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2b828b6c21ed422ba68418dabc3407a6: Network error: Client connection negotiation failed: client connection to 127.5.11.193:46087: connect: Connection refused (error 111)
I20260808 22:27:38.677812 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 23007
I20260808 22:27:38.691661 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 22438
2026-08-08T22:27:38Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2699 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:27:38 dist-test-slave-7f01 krb5kdc[23290](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:27:38 dist-test-slave-7f01 krb5kdc[23290](info): set up 2 sockets
Aug 08 22:27:38 dist-test-slave-7f01 krb5kdc[23290](info): commencing operation
krb5kdc: starting...
W20260808 22:27:39.958734 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:27:40 dist-test-slave-7f01 krb5kdc[23290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228060, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:27:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:27:40Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.5.11.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.5.11.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.5.11.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/rangeradmin_127.5.11.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.5.11.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/rangeradmin_127.5.11.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.5.11.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.5.11.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.5.11.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/rangerlookup_127.5.11.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.5.11.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/rangerlookup_127.5.11.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.11.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.5.11.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.11.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/HTTP_127.5.11.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.11.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/HTTP_127.5.11.212@KRBTEST.COM.keytab.
I20260808 22:27:40.126724 5167 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskM4Eo1i/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/postgres -l logfile start
2026-08-08 22:27:42.556 UTC [23339] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 22:27:42.556 UTC [23339] LOG: listening on IPv4 address "127.5.11.212", port 57855
2026-08-08 22:27:42.560 UTC [23339] LOG: listening on Unix socket "/tmp/.s.PGSQL.57855"
2026-08-08 22:27:42.596 UTC [23342] LOG: database system was shut down at 2026-08-08 22:27:41 UTC
2026-08-08 22:27:42.600 UTC [23339] LOG: database system is ready to accept connections
I20260808 22:27:43.759671 5167 mini_postgres.cc:96] Postgres bound to 57855
2026-08-08 22:27:43.768 UTC [23353] FATAL: database "slave" does not exist
127.5.11.212:57855 - accepting connections
I20260808 22:27:43.769194 5167 mini_ranger.cc:161] Starting Ranger...
I20260808 22:27:43.785274 5167 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 22:27:44.251365 5167 mini_ranger.cc:87] Created ranger Postgres database
I20260808 22:27:44.251492 5167 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ranger-admin
2026-08-08 22:27:44,574 [I] DB FLAVOR :POSTGRES
2026-08-08 22:27:44,574 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 22:27:44,574 [I] Checking connection..
2026-08-08 22:27:44,574 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:57855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 22:27:44,818 [I] Checking connection passed.
2026-08-08 22:27:44,819 [I] --------- Verifying version history table ---------
2026-08-08 22:27:44,819 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:57855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 22:27:45,047 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 22:27:45,047 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:57855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 22:27:45,289 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 22:27:45,290 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 22:27:45,290 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:57855/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 22:27:45.546 UTC [23447] WARNING: there is no transaction in progress
2026-08-08 22:27:45,553 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 22:27:45,553 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:57855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 22:27:45,814 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 22:27:45,814 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 22:27:45,815 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:57855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 22:27:46,060 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:57855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 22:27:46,317 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:57855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-7f01.c.gcp-upstream.internal','N') ;"
2026-08-08 22:27:46,556 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:57855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 22:27:46,781 [I] Table x_portal_user does not exist in database ranger
2026-08-08 22:27:46,781 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:57855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 22:27:47,022 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 22:27:47,022 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:57855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 22:27:47,283 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:57855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 22:27:47,541 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 22:27:47,541 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:57855/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 22:27:48.490 UTC [23604] WARNING: there is no transaction in progress
2026-08-08 22:27:48.504 UTC [23604] WARNING: there is no transaction in progress
2026-08-08 22:27:48.523 UTC [23604] WARNING: there is no transaction in progress
2026-08-08 22:27:48.539 UTC [23604] WARNING: there is no transaction in progress
2026-08-08 22:27:48.554 UTC [23604] WARNING: there is no transaction in progress
2026-08-08 22:27:48.570 UTC [23604] WARNING: there is no transaction in progress
2026-08-08 22:27:48.621 UTC [23604] WARNING: there is no transaction in progress
2026-08-08 22:27:48.627 UTC [23604] WARNING: there is no transaction in progress
2026-08-08 22:27:48.640 UTC [23604] WARNING: there is no transaction in progress
2026-08-08 22:27:48.649 UTC [23604] WARNING: there is no transaction in progress
2026-08-08 22:27:48.659 UTC [23604] WARNING: there is no transaction in progress
2026-08-08 22:27:48.671 UTC [23604] WARNING: there is no transaction in progress
2026-08-08 22:27:48.681 UTC [23604] WARNING: there is no transaction in progress
2026-08-08 22:27:48.686 UTC [23604] WARNING: there is no transaction in progress
2026-08-08 22:27:48.692 UTC [23604] WARNING: there is no transaction in progress
2026-08-08 22:27:48.698 UTC [23604] WARNING: there is no transaction in progress
2026-08-08 22:27:49,303 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 22:27:49,303 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:57855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 22:27:49,553 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 22:27:49,553 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:57855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 22:27:49,818 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 22:27:49,819 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:57855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 22:27:50,055 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 22:27:50,055 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:57855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 22:27:50,291 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:57855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 22:27:50,538 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:57855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-7f01.c.gcp-upstream.internal';"
2026-08-08 22:27:50,756 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 22:27:50,756 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:57855/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 22:27:51,012 [I] DB_PATCHES have already been applied
I20260808 22:27:51.018294 5167 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ranger-admin:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskM4Eo1i/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 22:27:51.018362 5167 mini_ranger.cc:193] Using host: 127.5.11.212
I20260808 22:27:51.023154 5167 mini_ranger.cc:227] Ranger admin URL: http://127.5.11.212:60419
Aug 08, 2026 10:27:51 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 10:27:51 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 10:27:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 10:27:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 10:27:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 10:27:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 10:27:51 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 22:27:52.428270 5167 mini_ranger.cc:160] Time spent starting Ranger: real 8.659s user 0.002s sys 0.010s
Aug 08, 2026 10:28:11 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 10:28:11 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 08, 2026 10:28:11 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 10:28:11 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 10:28:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 10:28:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 10:28:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 10:28:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 10:28:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 10:28:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 10:28:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 10:28:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 10:28:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 10:28:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 10:28:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 10:28:11 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 10:28:11 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 10:28:12 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 22:28:13.213516 5167 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.5.11.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.5.11.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.5.11.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/rangerkms_127.5.11.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.5.11.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/rangerkms_127.5.11.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.11.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.5.11.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.11.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/HTTP_127.5.11.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.11.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/HTTP_127.5.11.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 22:28:13 dist-test-slave-7f01 krb5kdc[23290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228093, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 22:28:13.304217 5167 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 22:28:13.314294 5167 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 22:28:14.181250 5167 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 22:28:14.181361 5167 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ranger-kms
I20260808 22:28:14.181464 5167 mini_ranger_kms.cc:227] Using postgres at 127.5.11.212:57855
2026-08-08 22:28:14,568 [I] DB FLAVOR :POSTGRES
2026-08-08 22:28:14,568 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 22:28:14,569 [I] Checking connection
2026-08-08 22:28:14,569 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:57855/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 22:28:14,816 [I] connection success
2026-08-08 22:28:14,816 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 22:28:14,816 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 22:28:14,816 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:57855/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 22:28:15,079 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 22:28:15,079 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 22:28:15,079 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 22:28:15,079 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:57855/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 22:28:15,340 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 22:28:15.812656 5167 mini_ranger_kms.cc:310] Created kms service
I20260808 22:28:16.040921 5167 mini_ranger_kms.cc:326] Created kudu user
I20260808 22:28:16.084777 5167 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 22:28:16 dist-test-slave-7f01 krb5kdc[23290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228093, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.5.11.212@KRBTEST.COM
I20260808 22:28:16.469127 5167 mini_ranger_kms.cc:384] Added ranger policy
I20260808 22:28:16.469301 5167 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ranger-kms:/tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 22:28:16.469322 5167 mini_ranger_kms.cc:240] Using host: 127.5.11.212
I20260808 22:28:16.474303 5167 mini_ranger_kms.cc:276] Ranger KMS PID: 23894
I20260808 22:28:16.474457 5167 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.5.11.212:35285
Aug 08, 2026 10:28:17 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 10:28:17 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 10:28:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 10:28:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 10:28:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 10:28:18 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 22:28:18 dist-test-slave-7f01 krb5kdc[23290](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786228098, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.11.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:28:19.185324 5167 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.881s user 0.010s sys 0.358s
I20260808 22:28:19.185465 5167 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 22:28:19.185585 5167 mini_ranger_kms.cc:401] 127.5.11.212:35285/kms/v1/keys
Aug 08 22:28:19 dist-test-slave-7f01 krb5kdc[23290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228098, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.11.212@KRBTEST.COM for HTTP/127.5.11.212@KRBTEST.COM
Aug 08 22:28:19 dist-test-slave-7f01 krb5kdc[23290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228098, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.11.212@KRBTEST.COM for HTTP/127.5.11.212@KRBTEST.COM
Aug 08 22:28:22 dist-test-slave-7f01 krb5kdc[23290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.5.11.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
I20260808 22:28:22.066803 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:35549
--webserver_interface=127.5.11.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:39983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:35549
--ranger_config_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.11.212:35285/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:28:22.195075 23977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:28:22.195351 23977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:28:22.195392 23977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:28:22.199366 23977 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:28:22.199425 23977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:28:22.199445 23977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:28:22.199462 23977 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:28:22.199477 23977 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:28:22.204778 23977 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:39983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/master-0/wal
--ranger_kms_url=127.5.11.212:35285/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:35549
--ranger_config_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:35549
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:28:22.205729 23977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:28:22.206701 23977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 22:28:22.215677 23977 server_base.cc:1034] running on GCE node
W20260808 22:28:22.215727 23983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:28:22.215780 23984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:28:22.215881 23987 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:28:22.216686 23977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:28:22.217823 23977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:28:22.219005 23977 hybrid_clock.cc:648] HybridClock initialized: now 1786228102218976 us; error 40 us; skew 500 ppm
Aug 08 22:28:22 dist-test-slave-7f01 krb5kdc[23290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228102, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:28:22.221928 23977 init.cc:377] Logged in from keytab as kudu/127.5.11.254@KRBTEST.COM (short username kudu)
I20260808 22:28:22.222805 23977 webserver.cc:460] Webserver started at http://127.5.11.254:46169/ using document root <none> and password file <none>
I20260808 22:28:22.223366 23977 fs_manager.cc:359] Metadata directory not provided
I20260808 22:28:22.223449 23977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:28:22.223687 23977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 22:28:22 dist-test-slave-7f01 krb5kdc[23290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228102, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for HTTP/127.5.11.212@KRBTEST.COM
I20260808 22:28:22.278501 23977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "a8b8d4ca74154845a2162cc9a04d241b"
format_stamp: "Formatted at 2026-08-08 22:28:22 on dist-test-slave-7f01"
server_key: "00e726af057c16371194ea2928aaa0f2"
server_key_iv: "f1b00b6993ca6f990af9eecf8c1acf0c"
server_key_version: "kuduclusterkey@0"
I20260808 22:28:22.279261 23977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "a8b8d4ca74154845a2162cc9a04d241b"
format_stamp: "Formatted at 2026-08-08 22:28:22 on dist-test-slave-7f01"
server_key: "00e726af057c16371194ea2928aaa0f2"
server_key_iv: "f1b00b6993ca6f990af9eecf8c1acf0c"
server_key_version: "kuduclusterkey@0"
I20260808 22:28:22.284569 23977 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260808 22:28:22.298648 23998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:28:22.300187 23977 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:28:22.300331 23977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "a8b8d4ca74154845a2162cc9a04d241b"
format_stamp: "Formatted at 2026-08-08 22:28:22 on dist-test-slave-7f01"
server_key: "00e726af057c16371194ea2928aaa0f2"
server_key_iv: "f1b00b6993ca6f990af9eecf8c1acf0c"
server_key_version: "kuduclusterkey@0"
I20260808 22:28:22.300474 23977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:28:22.318241 23977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:28:22.320960 23977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:28:22.321177 23977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:28:22.329711 23977 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:35549
I20260808 22:28:22.329739 24073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:35549 every 8 connection(s)
I20260808 22:28:22.330677 23977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:28:22.333529 24074 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:28:22.335602 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 23977
I20260808 22:28:22.335749 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:28:22.336079 5167 external_mini_cluster.cc:1428] Setting key 2acd0c852f563c1d3bbec00302808ad8
I20260808 22:28:22.340286 24074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8b8d4ca74154845a2162cc9a04d241b: Bootstrap starting.
I20260808 22:28:22.343250 24074 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8b8d4ca74154845a2162cc9a04d241b: Neither blocks nor log segments found. Creating new log.
I20260808 22:28:22.344190 24074 log.cc:824] T 00000000000000000000000000000000 P a8b8d4ca74154845a2162cc9a04d241b: Log is configured to *not* fsync() on all Append() calls
I20260808 22:28:22.346252 24074 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8b8d4ca74154845a2162cc9a04d241b: No bootstrap required, opened a new log
Aug 08 22:28:22 dist-test-slave-7f01 krb5kdc[23290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228102, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:28:22.350999 24074 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8b8d4ca74154845a2162cc9a04d241b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8b8d4ca74154845a2162cc9a04d241b" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 35549 } }
I20260808 22:28:22.351310 24074 raft_consensus.cc:374] T 00000000000000000000000000000000 P a8b8d4ca74154845a2162cc9a04d241b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:28:22.351373 24074 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8b8d4ca74154845a2162cc9a04d241b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8b8d4ca74154845a2162cc9a04d241b, State: Initialized, Role: FOLLOWER
I20260808 22:28:22.351895 24074 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8b8d4ca74154845a2162cc9a04d241b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8b8d4ca74154845a2162cc9a04d241b" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 35549 } }
I20260808 22:28:22.352057 24074 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8b8d4ca74154845a2162cc9a04d241b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:28:22.352159 24074 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8b8d4ca74154845a2162cc9a04d241b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:28:22.352267 24074 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8b8d4ca74154845a2162cc9a04d241b [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:28:22.353358 24074 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8b8d4ca74154845a2162cc9a04d241b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8b8d4ca74154845a2162cc9a04d241b" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 35549 } }
I20260808 22:28:22.353443 24079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:28:22.337546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41496 (local address 127.5.11.254:35549)
0808 22:28:22.337984 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0808 22:28:22.338006 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 22:28:22.338051 (+ 45us) server_negotiation.cc:407] Connection header received
0808 22:28:22.339108 (+ 1057us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:28:22.339133 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:28:22.339560 (+ 427us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:28:22.339937 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:28:22.341063 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:28:22.342755 (+ 1692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:28:22.343788 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:28:22.344483 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:28:22.344659 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:28:22.347965 (+ 3306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:28:22.347977 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:28:22.347996 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:28:22.348039 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:28:22.350123 (+ 2084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:28:22.350571 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:28:22.350577 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:28:22.350583 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:28:22.350717 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:28:22.350975 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:28:22.350979 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:28:22.350981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:28:22.351604 (+ 623us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:28:22.351809 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0808 22:28:22.352176 (+ 367us) server_negotiation.cc:299] Negotiation successful
0808 22:28:22.352480 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":124,"threads_started":1}
I20260808 22:28:22.353715 24074 leader_election.cc:302] T 00000000000000000000000000000000 P a8b8d4ca74154845a2162cc9a04d241b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8b8d4ca74154845a2162cc9a04d241b; no voters:
I20260808 22:28:22.354123 24074 leader_election.cc:288] T 00000000000000000000000000000000 P a8b8d4ca74154845a2162cc9a04d241b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:28:22.354192 24082 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8b8d4ca74154845a2162cc9a04d241b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:28:22.354487 24082 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8b8d4ca74154845a2162cc9a04d241b [term 1 LEADER]: Becoming Leader. State: Replica: a8b8d4ca74154845a2162cc9a04d241b, State: Running, Role: LEADER
I20260808 22:28:22.354836 24082 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8b8d4ca74154845a2162cc9a04d241b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8b8d4ca74154845a2162cc9a04d241b" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 35549 } }
I20260808 22:28:22.355486 24074 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8b8d4ca74154845a2162cc9a04d241b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:28:22.356325 24084 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8b8d4ca74154845a2162cc9a04d241b [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8b8d4ca74154845a2162cc9a04d241b. Latest consensus state: current_term: 1 leader_uuid: "a8b8d4ca74154845a2162cc9a04d241b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8b8d4ca74154845a2162cc9a04d241b" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 35549 } } }
I20260808 22:28:22.356562 24084 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8b8d4ca74154845a2162cc9a04d241b [sys.catalog]: This master's current role is: LEADER
I20260808 22:28:22.356853 24083 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8b8d4ca74154845a2162cc9a04d241b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a8b8d4ca74154845a2162cc9a04d241b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8b8d4ca74154845a2162cc9a04d241b" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 35549 } } }
I20260808 22:28:22.356978 24083 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8b8d4ca74154845a2162cc9a04d241b [sys.catalog]: This master's current role is: LEADER
I20260808 22:28:22.357441 24092 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:28:22.360335 24092 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:28:22.360878 24074 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 22:28:22.372720 24092 catalog_manager.cc:1347] Generated new cluster ID: b84f2bdda6314bb9b6553de558ca1b23
I20260808 22:28:22.372808 24092 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:28:22.402263 24092 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:28:22.404126 24092 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:28:22.415725 24092 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a8b8d4ca74154845a2162cc9a04d241b: Generated new TSK 0
I20260808 22:28:22.416637 24092 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.11.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab.
Entry for principal kudu/127.5.11.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab.
I20260808 22:28:23.131978 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.11.212:35285/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.193:0
--local_ip_for_outbound_sockets=127.5.11.193
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:35549
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:39983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:28:23.267592 24139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:28:23.267992 24139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:28:23.268091 24139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:28:23.272192 24139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:28:23.272264 24139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:28:23.272356 24139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.193
I20260808 22:28:23.277755 24139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:39983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.5.11.212:35285/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:35549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24139
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.193
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:28:23.278960 24139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:28:23.280108 24139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:28:23.290601 24148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:28:23.290700 24147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:28:23.290748 24150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:28:23.292171 24139 server_base.cc:1034] running on GCE node
I20260808 22:28:23.292796 24139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:28:23.293635 24139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:28:23.294855 24139 hybrid_clock.cc:648] HybridClock initialized: now 1786228103294828 us; error 45 us; skew 500 ppm
Aug 08 22:28:23 dist-test-slave-7f01 krb5kdc[23290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228103, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:28:23.298871 24139 init.cc:377] Logged in from keytab as kudu/127.5.11.193@KRBTEST.COM (short username kudu)
I20260808 22:28:23.300014 24139 webserver.cc:460] Webserver started at http://127.5.11.193:36583/ using document root <none> and password file <none>
I20260808 22:28:23.300768 24139 fs_manager.cc:359] Metadata directory not provided
I20260808 22:28:23.300875 24139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:28:23.301107 24139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 22:28:23 dist-test-slave-7f01 krb5kdc[23290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228103, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for HTTP/127.5.11.212@KRBTEST.COM
I20260808 22:28:23.318324 24139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-0/data/instance:
uuid: "7c4c33cc3b2b44b49f053070c1e3d253"
format_stamp: "Formatted at 2026-08-08 22:28:23 on dist-test-slave-7f01"
server_key: "16517c1c13ad6f83cc2e2ae219292fd2"
server_key_iv: "5421ad1dad240dfb26a523d70b2a406e"
server_key_version: "kuduclusterkey@0"
I20260808 22:28:23.319249 24139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance:
uuid: "7c4c33cc3b2b44b49f053070c1e3d253"
format_stamp: "Formatted at 2026-08-08 22:28:23 on dist-test-slave-7f01"
server_key: "16517c1c13ad6f83cc2e2ae219292fd2"
server_key_iv: "5421ad1dad240dfb26a523d70b2a406e"
server_key_version: "kuduclusterkey@0"
I20260808 22:28:23.324784 24139 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 22:28:23.353012 24165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:28:23.355180 24139 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260808 22:28:23.355393 24139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-0/wal
uuid: "7c4c33cc3b2b44b49f053070c1e3d253"
format_stamp: "Formatted at 2026-08-08 22:28:23 on dist-test-slave-7f01"
server_key: "16517c1c13ad6f83cc2e2ae219292fd2"
server_key_iv: "5421ad1dad240dfb26a523d70b2a406e"
server_key_version: "kuduclusterkey@0"
I20260808 22:28:23.355633 24139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:28:23.398924 24139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:28:23.402830 24139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:28:23.403139 24139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:28:23.404023 24139 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:28:23.405210 24139 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:28:23.405319 24139 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:28:23.405450 24139 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:28:23.405539 24139 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:28:23.421622 24139 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.193:39213
I20260808 22:28:23.422883 24139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
I20260808 22:28:23.423892 24348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.193:39213 every 8 connection(s)
I20260808 22:28:23.432713 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 24139
I20260808 22:28:23.432857 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance
I20260808 22:28:23.433172 5167 external_mini_cluster.cc:1428] Setting key 3c7b5636398745a9e60400c8330305f8
WARNING: no policy specified for kudu/127.5.11.194@KRBTEST.COM; defaulting to no policy
Aug 08 22:28:23 dist-test-slave-7f01 krb5kdc[23290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228103, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.194@KRBTEST.COM" created.
I20260808 22:28:23.454664 24354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:28:23.426806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:35549 (local address 127.5.11.193:46949)
0808 22:28:23.428714 (+ 1908us) negotiation.cc:107] Waiting for socket to connect
0808 22:28:23.428795 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0808 22:28:23.430468 (+ 1673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:28:23.431191 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:28:23.431201 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:28:23.431704 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:28:23.440655 (+ 8951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:28:23.440685 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:28:23.441971 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:28:23.441978 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:28:23.443765 (+ 1787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:28:23.443782 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:28:23.444754 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:28:23.444758 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:28:23.444997 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:28:23.445839 (+ 842us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:28:23.445863 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:28:23.448799 (+ 2936us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:28:23.451209 (+ 2410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:28:23.451219 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:28:23.451239 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:28:23.451633 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:28:23.452023 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:28:23.452029 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:28:23.452034 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:28:23.452243 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:28:23.452809 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:28:23.452817 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:28:23.453181 (+ 364us) client_negotiation.cc:769] Sending connection context
0808 22:28:23.453399 (+ 218us) client_negotiation.cc:240] Negotiation successful
0808 22:28:23.453777 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1620,"thread_start_us":153,"threads_started":1}
I20260808 22:28:23.455456 24355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:28:23.427967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:46949 (local address 127.5.11.254:35549)
0808 22:28:23.429379 (+ 1412us) server_negotiation.cc:206] Beginning negotiation
0808 22:28:23.429384 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:28:23.429414 (+ 30us) server_negotiation.cc:407] Connection header received
0808 22:28:23.430758 (+ 1344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:28:23.430762 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:28:23.430823 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:28:23.430946 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:28:23.440870 (+ 9924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:28:23.441791 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:28:23.443936 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:28:23.444375 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:28:23.444534 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:28:23.449004 (+ 4470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:28:23.449008 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:28:23.449010 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:28:23.449034 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:28:23.451053 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:28:23.451784 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:28:23.451791 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:28:23.451795 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:28:23.451909 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:28:23.452370 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:28:23.452375 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:28:23.452378 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:28:23.452647 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:28:23.452749 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 22:28:23.455167 (+ 2418us) server_negotiation.cc:299] Negotiation successful
0808 22:28:23.455306 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1324,"thread_start_us":99,"threads_started":1}
I20260808 22:28:23.457141 24349 heartbeater.cc:347] Connected to a master server at 127.5.11.254:35549
I20260808 22:28:23.457430 24349 heartbeater.cc:464] Registering TS with master...
I20260808 22:28:23.458122 24349 heartbeater.cc:511] Master 127.5.11.254:35549 requested a full tablet report, sending...
I20260808 22:28:23.460286 24019 ts_manager.cc:194] Registered new tserver with Master: 7c4c33cc3b2b44b49f053070c1e3d253 (127.5.11.193:39213)
I20260808 22:28:23.462164 24019 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.193@KRBTEST.COM'} at 127.5.11.193:46949
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab.
Entry for principal kudu/127.5.11.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab.
I20260808 22:28:23.471045 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.11.212:35285/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.194:0
--local_ip_for_outbound_sockets=127.5.11.194
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:35549
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:39983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 22:28:23 dist-test-slave-7f01 krb5kdc[23290](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786228103, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260808 22:28:23.644806 24360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:28:23.645078 24360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:28:23.645123 24360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:28:23.649307 24360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:28:23.649374 24360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:28:23.649431 24360 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.194
I20260808 22:28:23.655287 24360 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:39983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.5.11.212:35285/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:35549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.194
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:28:23.656476 24360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:28:23.657624 24360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:28:23.669325 24372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:28:23.669478 24370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:28:23.669355 24374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:28:23.670151 24360 server_base.cc:1034] running on GCE node
I20260808 22:28:23.670619 24360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:28:23.671588 24360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:28:23.672847 24360 hybrid_clock.cc:648] HybridClock initialized: now 1786228103672823 us; error 35 us; skew 500 ppm
Aug 08 22:28:23 dist-test-slave-7f01 krb5kdc[23290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228103, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:28:23.676683 24360 init.cc:377] Logged in from keytab as kudu/127.5.11.194@KRBTEST.COM (short username kudu)
I20260808 22:28:23.677754 24360 webserver.cc:460] Webserver started at http://127.5.11.194:41009/ using document root <none> and password file <none>
I20260808 22:28:23.678470 24360 fs_manager.cc:359] Metadata directory not provided
I20260808 22:28:23.678596 24360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:28:23.678876 24360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 22:28:23 dist-test-slave-7f01 krb5kdc[23290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228103, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for HTTP/127.5.11.212@KRBTEST.COM
I20260808 22:28:23.694499 24360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-1/data/instance:
uuid: "b066d92a71184add8cfe4603f4ba82ff"
format_stamp: "Formatted at 2026-08-08 22:28:23 on dist-test-slave-7f01"
server_key: "a72917b1c085295d6cc3f6266a642362"
server_key_iv: "9fc4ebcd3110827dcb593684f72aa542"
server_key_version: "kuduclusterkey@0"
I20260808 22:28:23.695137 24360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance:
uuid: "b066d92a71184add8cfe4603f4ba82ff"
format_stamp: "Formatted at 2026-08-08 22:28:23 on dist-test-slave-7f01"
server_key: "a72917b1c085295d6cc3f6266a642362"
server_key_iv: "9fc4ebcd3110827dcb593684f72aa542"
server_key_version: "kuduclusterkey@0"
I20260808 22:28:23.699921 24360 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.001s
I20260808 22:28:23.716143 24384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:28:23.717602 24360 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260808 22:28:23.717753 24360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-1/wal
uuid: "b066d92a71184add8cfe4603f4ba82ff"
format_stamp: "Formatted at 2026-08-08 22:28:23 on dist-test-slave-7f01"
server_key: "a72917b1c085295d6cc3f6266a642362"
server_key_iv: "9fc4ebcd3110827dcb593684f72aa542"
server_key_version: "kuduclusterkey@0"
I20260808 22:28:23.717900 24360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:28:23.740499 24360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:28:23.744767 24360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:28:23.745040 24360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:28:23.745985 24360 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:28:23.747408 24360 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:28:23.747540 24360 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:28:23.747668 24360 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:28:23.747808 24360 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:28:23.774382 24360 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.194:43565
I20260808 22:28:23.775612 24360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
I20260808 22:28:23.776000 24554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.194:43565 every 8 connection(s)
I20260808 22:28:23.782904 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 24360
I20260808 22:28:23.783049 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance
I20260808 22:28:23.783339 5167 external_mini_cluster.cc:1428] Setting key 8d033d9beaaf037746e9dc0c404e0948
Aug 08 22:28:23 dist-test-slave-7f01 krb5kdc[23290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228103, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:28:23.794672 24355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:28:23.781272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:59689 (local address 127.5.11.254:35549)
0808 22:28:23.781412 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0808 22:28:23.781417 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:28:23.781435 (+ 18us) server_negotiation.cc:407] Connection header received
0808 22:28:23.782033 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:28:23.782037 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:28:23.782095 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:28:23.782198 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:28:23.783887 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:28:23.784738 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:28:23.786456 (+ 1718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:28:23.786837 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:28:23.786919 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:28:23.790012 (+ 3093us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:28:23.790017 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:28:23.790019 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:28:23.790046 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:28:23.791913 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:28:23.792669 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:28:23.792673 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:28:23.792675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:28:23.792786 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:28:23.793265 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:28:23.793271 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:28:23.793273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:28:23.793522 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:28:23.793606 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0808 22:28:23.794338 (+ 732us) server_negotiation.cc:299] Negotiation successful
0808 22:28:23.794497 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 22:28:23.796797 24557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:28:23.779923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:35549 (local address 127.5.11.194:59689)
0808 22:28:23.780618 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0808 22:28:23.780676 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 22:28:23.781737 (+ 1061us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:28:23.782406 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:28:23.782415 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:28:23.782889 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:28:23.783676 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:28:23.783698 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:28:23.784921 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:28:23.784925 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:28:23.786311 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:28:23.786323 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:28:23.787117 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:28:23.787121 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:28:23.787314 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:28:23.788068 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:28:23.788089 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:28:23.789825 (+ 1736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:28:23.792099 (+ 2274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:28:23.792106 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:28:23.792123 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:28:23.792531 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:28:23.792923 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:28:23.792926 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:28:23.792928 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:28:23.793118 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:28:23.793702 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:28:23.793708 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:28:23.794031 (+ 323us) client_negotiation.cc:769] Sending connection context
0808 22:28:23.795143 (+ 1112us) client_negotiation.cc:240] Negotiation successful
0808 22:28:23.795543 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":247,"threads_started":1}
I20260808 22:28:23.798589 24555 heartbeater.cc:347] Connected to a master server at 127.5.11.254:35549
I20260808 22:28:23.798840 24555 heartbeater.cc:464] Registering TS with master...
I20260808 22:28:23.799530 24555 heartbeater.cc:511] Master 127.5.11.254:35549 requested a full tablet report, sending...
I20260808 22:28:23.800889 24019 ts_manager.cc:194] Registered new tserver with Master: b066d92a71184add8cfe4603f4ba82ff (127.5.11.194:43565)
I20260808 22:28:23.801725 24019 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.194@KRBTEST.COM'} at 127.5.11.194:59689
WARNING: no policy specified for kudu/127.5.11.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab.
Entry for principal kudu/127.5.11.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab.
I20260808 22:28:23.835039 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.11.212:35285/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.195:0
--local_ip_for_outbound_sockets=127.5.11.195
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:35549
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:39983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:28:23.998668 24560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:28:23.998952 24560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:28:23.999060 24560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:28:24.003546 24560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:28:24.003787 24560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:28:24.003918 24560 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.195
I20260808 22:28:24.010414 24560 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:39983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.5.11.212:35285/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:35549
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24560
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.195
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:28:24.011620 24560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:28:24.012686 24560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:28:24.024128 24569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:28:24.024048 24573 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:28:24.024519 24576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:28:24.025295 24560 server_base.cc:1034] running on GCE node
I20260808 22:28:24.025750 24560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:28:24.026515 24560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:28:24.027695 24560 hybrid_clock.cc:648] HybridClock initialized: now 1786228104027655 us; error 50 us; skew 500 ppm
Aug 08 22:28:24 dist-test-slave-7f01 krb5kdc[23290](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228104, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:28:24.030815 24560 init.cc:377] Logged in from keytab as kudu/127.5.11.195@KRBTEST.COM (short username kudu)
I20260808 22:28:24.031912 24560 webserver.cc:460] Webserver started at http://127.5.11.195:35741/ using document root <none> and password file <none>
I20260808 22:28:24.032704 24560 fs_manager.cc:359] Metadata directory not provided
I20260808 22:28:24.032820 24560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:28:24.033114 24560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 22:28:24 dist-test-slave-7f01 krb5kdc[23290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228104, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for HTTP/127.5.11.212@KRBTEST.COM
Aug 08 22:28:24 dist-test-slave-7f01 krb5kdc[23290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228103, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for HTTP/127.5.11.212@KRBTEST.COM
I20260808 22:28:24.072728 24560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-2/data/instance:
uuid: "af3e13cfbd2f46d79292646cebbc89e7"
format_stamp: "Formatted at 2026-08-08 22:28:24 on dist-test-slave-7f01"
server_key: "da5da43a3dacccb4cd41845f350f782f"
server_key_iv: "290e30bddd4742cfbb2c069194b20801"
server_key_version: "kuduclusterkey@0"
I20260808 22:28:24.073691 24560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance:
uuid: "af3e13cfbd2f46d79292646cebbc89e7"
format_stamp: "Formatted at 2026-08-08 22:28:24 on dist-test-slave-7f01"
server_key: "da5da43a3dacccb4cd41845f350f782f"
server_key_iv: "290e30bddd4742cfbb2c069194b20801"
server_key_version: "kuduclusterkey@0"
I20260808 22:28:24.079533 24560 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260808 22:28:24.118731 24584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:28:24.122822 24560 fs_manager.cc:725] Time spent opening block manager: real 0.023s user 0.004s sys 0.000s
I20260808 22:28:24.123919 24560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-2/wal
uuid: "af3e13cfbd2f46d79292646cebbc89e7"
format_stamp: "Formatted at 2026-08-08 22:28:24 on dist-test-slave-7f01"
server_key: "da5da43a3dacccb4cd41845f350f782f"
server_key_iv: "290e30bddd4742cfbb2c069194b20801"
server_key_version: "kuduclusterkey@0"
I20260808 22:28:24.126068 24560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:28:24.145569 24560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:28:24.149274 24560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:28:24.149533 24560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:28:24.150354 24560 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:28:24.151520 24560 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:28:24.151614 24560 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:28:24.151696 24560 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:28:24.151762 24560 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:28:24.196961 24560 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.195:33517
I20260808 22:28:24.197121 24764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.195:33517 every 8 connection(s)
I20260808 22:28:24.198083 24560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
I20260808 22:28:24.205899 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 24560
I20260808 22:28:24.206032 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance
I20260808 22:28:24.206393 5167 external_mini_cluster.cc:1428] Setting key f0778e101786e69ee76bae751f255205
Aug 08 22:28:24 dist-test-slave-7f01 krb5kdc[23290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228104, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:28:24.220000 24355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:28:24.207131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:45105 (local address 127.5.11.254:35549)
0808 22:28:24.207255 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 22:28:24.207260 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:28:24.207274 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:28:24.207615 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:28:24.207621 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:28:24.207685 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:28:24.207762 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:28:24.210208 (+ 2446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:28:24.210748 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:28:24.212051 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:28:24.212286 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:28:24.212403 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:28:24.215309 (+ 2906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:28:24.215312 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:28:24.215315 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:28:24.215338 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:28:24.216991 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:28:24.217545 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:28:24.217548 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:28:24.217550 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:28:24.217631 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:28:24.218008 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:28:24.218012 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:28:24.218013 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:28:24.218233 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:28:24.218342 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 22:28:24.218886 (+ 544us) server_negotiation.cc:299] Negotiation successful
0808 22:28:24.219024 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 22:28:24.221395 24768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:28:24.204416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:35549 (local address 127.5.11.195:45105)
0808 22:28:24.205217 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0808 22:28:24.205380 (+ 163us) client_negotiation.cc:174] Beginning negotiation
0808 22:28:24.206892 (+ 1512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:28:24.208740 (+ 1848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:28:24.208749 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:28:24.209386 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:28:24.210041 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:28:24.210058 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:28:24.210885 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:28:24.210888 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:28:24.211929 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:28:24.211940 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:28:24.212456 (+ 516us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:28:24.212461 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:28:24.212694 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:28:24.213424 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:28:24.213441 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:28:24.215131 (+ 1690us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:28:24.217121 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:28:24.217127 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:28:24.217142 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:28:24.217426 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:28:24.217738 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:28:24.217741 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:28:24.217743 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:28:24.217901 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:28:24.218350 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:28:24.218359 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:28:24.218628 (+ 269us) client_negotiation.cc:769] Sending connection context
0808 22:28:24.218817 (+ 189us) client_negotiation.cc:240] Negotiation successful
0808 22:28:24.219144 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":269,"threads_started":1}
I20260808 22:28:24.222802 24766 heartbeater.cc:347] Connected to a master server at 127.5.11.254:35549
I20260808 22:28:24.223080 24766 heartbeater.cc:464] Registering TS with master...
I20260808 22:28:24.223692 24766 heartbeater.cc:511] Master 127.5.11.254:35549 requested a full tablet report, sending...
I20260808 22:28:24.226497 24019 ts_manager.cc:194] Registered new tserver with Master: af3e13cfbd2f46d79292646cebbc89e7 (127.5.11.195:33517)
I20260808 22:28:24.227130 24019 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.195@KRBTEST.COM'} at 127.5.11.195:45105
I20260808 22:28:24.233191 5167 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:28:24.243356 24355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:28:24.235359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41506 (local address 127.5.11.254:35549)
0808 22:28:24.235507 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 22:28:24.235515 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 22:28:24.235815 (+ 300us) server_negotiation.cc:407] Connection header received
0808 22:28:24.235976 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:28:24.235979 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:28:24.236023 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:28:24.236116 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:28:24.236817 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:28:24.237341 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:28:24.238208 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:28:24.238443 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:28:24.238553 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:28:24.240102 (+ 1549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:28:24.240107 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:28:24.240109 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:28:24.240136 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:28:24.241627 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:28:24.242078 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:28:24.242081 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:28:24.242083 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:28:24.242168 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:28:24.242454 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:28:24.242458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:28:24.242460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:28:24.242641 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:28:24.242791 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 22:28:24.243043 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 22:28:24.243154 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 22:28:24.246294 24019 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41506:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:28:24.249722 24019 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:28:24.270246 24784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:28:24.260012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:39213 (local address 127.0.0.1:43924)
0808 22:28:24.260795 (+ 783us) negotiation.cc:107] Waiting for socket to connect
0808 22:28:24.260862 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 22:28:24.260998 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:28:24.264252 (+ 3254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:28:24.264259 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:28:24.264281 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:28:24.264462 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:28:24.264471 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:28:24.266066 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:28:24.266070 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:28:24.267809 (+ 1739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:28:24.267830 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:28:24.269657 (+ 1827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:28:24.269661 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:28:24.269829 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:28:24.269877 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 22:28:24.269968 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 22:28:24.270091 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560,"thread_start_us":156,"threads_started":1}
I20260808 22:28:24.271319 24785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:28:24.260969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43924 (local address 127.5.11.193:39213)
0808 22:28:24.263781 (+ 2812us) server_negotiation.cc:206] Beginning negotiation
0808 22:28:24.263788 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:28:24.263818 (+ 30us) server_negotiation.cc:407] Connection header received
0808 22:28:24.263893 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:28:24.263898 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:28:24.264093 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:28:24.264616 (+ 523us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:28:24.264764 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:28:24.265935 (+ 1171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:28:24.268052 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:28:24.269397 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:28:24.270881 (+ 1484us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:28:24.271029 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 22:28:24.271079 (+ 50us) server_negotiation.cc:299] Negotiation successful
0808 22:28:24.271167 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2695,"thread_start_us":102,"threads_started":1}
I20260808 22:28:24.274160 24786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:28:24.261982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:33517 (local address 127.0.0.1:59048)
0808 22:28:24.266861 (+ 4879us) negotiation.cc:107] Waiting for socket to connect
0808 22:28:24.266876 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:28:24.267076 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:28:24.268561 (+ 1485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:28:24.268564 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:28:24.268581 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:28:24.268778 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:28:24.268786 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:28:24.271126 (+ 2340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:28:24.271129 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:28:24.272333 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:28:24.272346 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:28:24.273725 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:28:24.273728 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:28:24.273846 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:28:24.273871 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 22:28:24.273949 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 22:28:24.274019 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4729,"thread_start_us":144,"threads_started":1}
I20260808 22:28:24.274679 24241 tablet_service.cc:1467] Processing CreateTablet for tablet 23dfa8e2b1e2409f95ab0f7c41ea4896 (DEFAULT_TABLE table=test-table [id=2f2fbcda1ede4c61b5fb5b88b297eaa9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:28:24.275398 24791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:28:24.262505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59048 (local address 127.5.11.195:33517)
0808 22:28:24.263266 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0808 22:28:24.263289 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0808 22:28:24.266998 (+ 3709us) server_negotiation.cc:407] Connection header received
0808 22:28:24.267225 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:28:24.267231 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:28:24.268138 (+ 907us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:28:24.268376 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:28:24.268933 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:28:24.270974 (+ 2041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:28:24.272516 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:28:24.273367 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:28:24.273461 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:28:24.273611 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0808 22:28:24.274332 (+ 721us) server_negotiation.cc:299] Negotiation successful
0808 22:28:24.275252 (+ 920us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":155,"threads_started":1}
I20260808 22:28:24.276367 24241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23dfa8e2b1e2409f95ab0f7c41ea4896. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:28:24.277084 24787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:28:24.261762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:43565 (local address 127.0.0.1:43286)
0808 22:28:24.263434 (+ 1672us) negotiation.cc:107] Waiting for socket to connect
0808 22:28:24.263448 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:28:24.263544 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:28:24.264345 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:28:24.264349 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:28:24.264368 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:28:24.264607 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:28:24.264617 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:28:24.274417 (+ 9800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:28:24.274420 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:28:24.275501 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:28:24.275513 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:28:24.276737 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:28:24.276741 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:28:24.276836 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:28:24.276859 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 22:28:24.276911 (+ 52us) client_negotiation.cc:240] Negotiation successful
0808 22:28:24.276969 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1611,"thread_start_us":221,"threads_started":1}
I20260808 22:28:24.277689 24788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:28:24.262333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43286 (local address 127.5.11.194:43565)
0808 22:28:24.263555 (+ 1222us) server_negotiation.cc:206] Beginning negotiation
0808 22:28:24.263613 (+ 58us) server_negotiation.cc:399] Waiting for connection header
0808 22:28:24.263640 (+ 27us) server_negotiation.cc:407] Connection header received
0808 22:28:24.263719 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:28:24.263725 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:28:24.263978 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:28:24.264176 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:28:24.264772 (+ 596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:28:24.274259 (+ 9487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:28:24.275697 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:28:24.276598 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:28:24.277270 (+ 672us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:28:24.277418 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 22:28:24.277470 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 22:28:24.277554 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1101,"thread_start_us":159,"threads_started":1}
I20260808 22:28:24.278445 24659 tablet_service.cc:1467] Processing CreateTablet for tablet 23dfa8e2b1e2409f95ab0f7c41ea4896 (DEFAULT_TABLE table=test-table [id=2f2fbcda1ede4c61b5fb5b88b297eaa9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:28:24.279500 24659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23dfa8e2b1e2409f95ab0f7c41ea4896. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:28:24.283006 24451 tablet_service.cc:1467] Processing CreateTablet for tablet 23dfa8e2b1e2409f95ab0f7c41ea4896 (DEFAULT_TABLE table=test-table [id=2f2fbcda1ede4c61b5fb5b88b297eaa9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:28:24.284054 24451 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 23dfa8e2b1e2409f95ab0f7c41ea4896. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:28:24.285604 24794 tablet_bootstrap.cc:492] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P 7c4c33cc3b2b44b49f053070c1e3d253: Bootstrap starting.
I20260808 22:28:24.288048 24794 tablet_bootstrap.cc:654] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P 7c4c33cc3b2b44b49f053070c1e3d253: Neither blocks nor log segments found. Creating new log.
I20260808 22:28:24.288929 24794 log.cc:824] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P 7c4c33cc3b2b44b49f053070c1e3d253: Log is configured to *not* fsync() on all Append() calls
I20260808 22:28:24.296681 24796 tablet_bootstrap.cc:492] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P b066d92a71184add8cfe4603f4ba82ff: Bootstrap starting.
I20260808 22:28:24.297595 24794 tablet_bootstrap.cc:492] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P 7c4c33cc3b2b44b49f053070c1e3d253: No bootstrap required, opened a new log
I20260808 22:28:24.298044 24794 ts_tablet_manager.cc:1397] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P 7c4c33cc3b2b44b49f053070c1e3d253: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.000s
I20260808 22:28:24.300940 24796 tablet_bootstrap.cc:654] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P b066d92a71184add8cfe4603f4ba82ff: Neither blocks nor log segments found. Creating new log.
I20260808 22:28:24.302529 24796 log.cc:824] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P b066d92a71184add8cfe4603f4ba82ff: Log is configured to *not* fsync() on all Append() calls
I20260808 22:28:24.305258 24796 tablet_bootstrap.cc:492] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P b066d92a71184add8cfe4603f4ba82ff: No bootstrap required, opened a new log
I20260808 22:28:24.305199 24794 raft_consensus.cc:348] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P 7c4c33cc3b2b44b49f053070c1e3d253 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c4c33cc3b2b44b49f053070c1e3d253" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39213 } } peers { permanent_uuid: "b066d92a71184add8cfe4603f4ba82ff" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 43565 } } peers { permanent_uuid: "af3e13cfbd2f46d79292646cebbc89e7" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 33517 } }
I20260808 22:28:24.305473 24796 ts_tablet_manager.cc:1397] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P b066d92a71184add8cfe4603f4ba82ff: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260808 22:28:24.305486 24794 raft_consensus.cc:374] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P 7c4c33cc3b2b44b49f053070c1e3d253 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:28:24.305541 24794 raft_consensus.cc:729] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P 7c4c33cc3b2b44b49f053070c1e3d253 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c4c33cc3b2b44b49f053070c1e3d253, State: Initialized, Role: FOLLOWER
I20260808 22:28:24.306080 24794 consensus_queue.cc:260] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P 7c4c33cc3b2b44b49f053070c1e3d253 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c4c33cc3b2b44b49f053070c1e3d253" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39213 } } peers { permanent_uuid: "b066d92a71184add8cfe4603f4ba82ff" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 43565 } } peers { permanent_uuid: "af3e13cfbd2f46d79292646cebbc89e7" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 33517 } }
I20260808 22:28:24.307457 24794 ts_tablet_manager.cc:1428] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P 7c4c33cc3b2b44b49f053070c1e3d253: Time spent starting tablet: real 0.009s user 0.008s sys 0.000s
I20260808 22:28:24.307979 24349 heartbeater.cc:503] Master 127.5.11.254:35549 was elected leader, sending a full tablet report...
I20260808 22:28:24.308456 24797 tablet_bootstrap.cc:492] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P af3e13cfbd2f46d79292646cebbc89e7: Bootstrap starting.
I20260808 22:28:24.311519 24796 raft_consensus.cc:348] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P b066d92a71184add8cfe4603f4ba82ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c4c33cc3b2b44b49f053070c1e3d253" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39213 } } peers { permanent_uuid: "b066d92a71184add8cfe4603f4ba82ff" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 43565 } } peers { permanent_uuid: "af3e13cfbd2f46d79292646cebbc89e7" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 33517 } }
I20260808 22:28:24.312528 24797 tablet_bootstrap.cc:654] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P af3e13cfbd2f46d79292646cebbc89e7: Neither blocks nor log segments found. Creating new log.
I20260808 22:28:24.312557 24796 raft_consensus.cc:374] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P b066d92a71184add8cfe4603f4ba82ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:28:24.312613 24796 raft_consensus.cc:729] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P b066d92a71184add8cfe4603f4ba82ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b066d92a71184add8cfe4603f4ba82ff, State: Initialized, Role: FOLLOWER
I20260808 22:28:24.313285 24796 consensus_queue.cc:260] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P b066d92a71184add8cfe4603f4ba82ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c4c33cc3b2b44b49f053070c1e3d253" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39213 } } peers { permanent_uuid: "b066d92a71184add8cfe4603f4ba82ff" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 43565 } } peers { permanent_uuid: "af3e13cfbd2f46d79292646cebbc89e7" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 33517 } }
I20260808 22:28:24.313992 24797 log.cc:824] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P af3e13cfbd2f46d79292646cebbc89e7: Log is configured to *not* fsync() on all Append() calls
I20260808 22:28:24.314525 24796 ts_tablet_manager.cc:1428] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P b066d92a71184add8cfe4603f4ba82ff: Time spent starting tablet: real 0.009s user 0.004s sys 0.005s
I20260808 22:28:24.314961 24555 heartbeater.cc:503] Master 127.5.11.254:35549 was elected leader, sending a full tablet report...
I20260808 22:28:24.316715 24797 tablet_bootstrap.cc:492] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P af3e13cfbd2f46d79292646cebbc89e7: No bootstrap required, opened a new log
I20260808 22:28:24.316921 24797 ts_tablet_manager.cc:1397] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P af3e13cfbd2f46d79292646cebbc89e7: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.006s
I20260808 22:28:24.323191 24797 raft_consensus.cc:348] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P af3e13cfbd2f46d79292646cebbc89e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c4c33cc3b2b44b49f053070c1e3d253" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39213 } } peers { permanent_uuid: "b066d92a71184add8cfe4603f4ba82ff" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 43565 } } peers { permanent_uuid: "af3e13cfbd2f46d79292646cebbc89e7" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 33517 } }
I20260808 22:28:24.323459 24797 raft_consensus.cc:374] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P af3e13cfbd2f46d79292646cebbc89e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:28:24.323514 24797 raft_consensus.cc:729] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P af3e13cfbd2f46d79292646cebbc89e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af3e13cfbd2f46d79292646cebbc89e7, State: Initialized, Role: FOLLOWER
I20260808 22:28:24.323980 24797 consensus_queue.cc:260] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P af3e13cfbd2f46d79292646cebbc89e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c4c33cc3b2b44b49f053070c1e3d253" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39213 } } peers { permanent_uuid: "b066d92a71184add8cfe4603f4ba82ff" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 43565 } } peers { permanent_uuid: "af3e13cfbd2f46d79292646cebbc89e7" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 33517 } }
I20260808 22:28:24.325012 24797 ts_tablet_manager.cc:1428] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P af3e13cfbd2f46d79292646cebbc89e7: Time spent starting tablet: real 0.008s user 0.007s sys 0.001s
I20260808 22:28:24.325440 24766 heartbeater.cc:503] Master 127.5.11.254:35549 was elected leader, sending a full tablet report...
W20260808 22:28:24.425469 24352 tablet.cc:2369] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P 7c4c33cc3b2b44b49f053070c1e3d253: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 22:28:24.449491 24767 tablet.cc:2369] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P af3e13cfbd2f46d79292646cebbc89e7: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 08 22:28:24 dist-test-slave-7f01 krb5kdc[23290](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228103, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for HTTP/127.5.11.212@KRBTEST.COM
I20260808 22:28:24.489262 24806 raft_consensus.cc:482] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P af3e13cfbd2f46d79292646cebbc89e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:28:24.489614 24806 raft_consensus.cc:504] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P af3e13cfbd2f46d79292646cebbc89e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c4c33cc3b2b44b49f053070c1e3d253" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39213 } } peers { permanent_uuid: "b066d92a71184add8cfe4603f4ba82ff" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 43565 } } peers { permanent_uuid: "af3e13cfbd2f46d79292646cebbc89e7" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 33517 } }
I20260808 22:28:24.491223 24806 leader_election.cc:288] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P af3e13cfbd2f46d79292646cebbc89e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c4c33cc3b2b44b49f053070c1e3d253 (127.5.11.193:39213), b066d92a71184add8cfe4603f4ba82ff (127.5.11.194:43565)
I20260808 22:28:24.496341 24768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:28:24.491509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:39213 (local address 127.5.11.195:51943)
0808 22:28:24.491901 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0808 22:28:24.491919 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 22:28:24.492022 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:28:24.492508 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:28:24.492511 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:28:24.492532 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:28:24.492796 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:28:24.492808 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:28:24.493764 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:28:24.493770 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:28:24.494745 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:28:24.494757 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:28:24.495774 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:28:24.495778 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:28:24.495974 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:28:24.496010 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 22:28:24.496095 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 22:28:24.496180 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":1}
I20260808 22:28:24.496821 24785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:28:24.491952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:51943 (local address 127.5.11.193:39213)
0808 22:28:24.492241 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0808 22:28:24.492246 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:28:24.492262 (+ 16us) server_negotiation.cc:407] Connection header received
0808 22:28:24.492334 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:28:24.492338 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:28:24.492404 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:28:24.492486 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:28:24.492958 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:28:24.493678 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:28:24.494979 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:28:24.495603 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:28:24.495728 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:28:24.495810 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 22:28:24.496623 (+ 813us) server_negotiation.cc:299] Negotiation successful
0808 22:28:24.496702 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196}
I20260808 22:28:24.497532 24274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23dfa8e2b1e2409f95ab0f7c41ea4896" candidate_uuid: "af3e13cfbd2f46d79292646cebbc89e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c4c33cc3b2b44b49f053070c1e3d253" is_pre_election: true
I20260808 22:28:24.497759 24788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:28:24.492091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:42621 (local address 127.5.11.194:43565)
0808 22:28:24.492705 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0808 22:28:24.492710 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:28:24.493181 (+ 471us) server_negotiation.cc:407] Connection header received
0808 22:28:24.493281 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:28:24.493284 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:28:24.493340 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:28:24.493430 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:28:24.493721 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:28:24.494480 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:28:24.496144 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:28:24.496767 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:28:24.496946 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:28:24.496992 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 22:28:24.497557 (+ 565us) server_negotiation.cc:299] Negotiation successful
0808 22:28:24.497636 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491}
I20260808 22:28:24.497888 24274 raft_consensus.cc:2454] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P 7c4c33cc3b2b44b49f053070c1e3d253 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af3e13cfbd2f46d79292646cebbc89e7 in term 0.
I20260808 22:28:24.498126 24808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:28:24.491576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:43565 (local address 127.5.11.195:42621)
0808 22:28:24.493074 (+ 1498us) negotiation.cc:107] Waiting for socket to connect
0808 22:28:24.493087 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:28:24.493189 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:28:24.493463 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:28:24.493466 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:28:24.493483 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:28:24.493595 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:28:24.493600 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:28:24.494619 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:28:24.494623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:28:24.496003 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:28:24.496014 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:28:24.497292 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:28:24.497296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:28:24.497403 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:28:24.497426 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 22:28:24.497898 (+ 472us) client_negotiation.cc:240] Negotiation successful
0808 22:28:24.497978 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1436,"thread_start_us":1188,"threads_started":1}
I20260808 22:28:24.498469 24587 leader_election.cc:302] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P af3e13cfbd2f46d79292646cebbc89e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c4c33cc3b2b44b49f053070c1e3d253, af3e13cfbd2f46d79292646cebbc89e7; no voters:
I20260808 22:28:24.498740 24806 raft_consensus.cc:2781] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P af3e13cfbd2f46d79292646cebbc89e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:28:24.498836 24806 raft_consensus.cc:482] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P af3e13cfbd2f46d79292646cebbc89e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:28:24.498891 24806 raft_consensus.cc:3037] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P af3e13cfbd2f46d79292646cebbc89e7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:28:24.498842 24487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23dfa8e2b1e2409f95ab0f7c41ea4896" candidate_uuid: "af3e13cfbd2f46d79292646cebbc89e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b066d92a71184add8cfe4603f4ba82ff" is_pre_election: true
I20260808 22:28:24.499120 24487 raft_consensus.cc:2454] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P b066d92a71184add8cfe4603f4ba82ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate af3e13cfbd2f46d79292646cebbc89e7 in term 0.
I20260808 22:28:24.500808 24806 raft_consensus.cc:504] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P af3e13cfbd2f46d79292646cebbc89e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c4c33cc3b2b44b49f053070c1e3d253" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39213 } } peers { permanent_uuid: "b066d92a71184add8cfe4603f4ba82ff" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 43565 } } peers { permanent_uuid: "af3e13cfbd2f46d79292646cebbc89e7" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 33517 } }
I20260808 22:28:24.501279 24806 leader_election.cc:288] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P af3e13cfbd2f46d79292646cebbc89e7 [CANDIDATE]: Term 1 election: Requested vote from peers 7c4c33cc3b2b44b49f053070c1e3d253 (127.5.11.193:39213), b066d92a71184add8cfe4603f4ba82ff (127.5.11.194:43565)
I20260808 22:28:24.501813 24487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23dfa8e2b1e2409f95ab0f7c41ea4896" candidate_uuid: "af3e13cfbd2f46d79292646cebbc89e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b066d92a71184add8cfe4603f4ba82ff"
I20260808 22:28:24.501950 24487 raft_consensus.cc:3037] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P b066d92a71184add8cfe4603f4ba82ff [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:28:24.502053 24274 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "23dfa8e2b1e2409f95ab0f7c41ea4896" candidate_uuid: "af3e13cfbd2f46d79292646cebbc89e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c4c33cc3b2b44b49f053070c1e3d253"
I20260808 22:28:24.502185 24274 raft_consensus.cc:3037] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P 7c4c33cc3b2b44b49f053070c1e3d253 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:28:24.503202 24274 raft_consensus.cc:2454] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P 7c4c33cc3b2b44b49f053070c1e3d253 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af3e13cfbd2f46d79292646cebbc89e7 in term 1.
I20260808 22:28:24.503495 24487 raft_consensus.cc:2454] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P b066d92a71184add8cfe4603f4ba82ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate af3e13cfbd2f46d79292646cebbc89e7 in term 1.
I20260808 22:28:24.503520 24587 leader_election.cc:302] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P af3e13cfbd2f46d79292646cebbc89e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c4c33cc3b2b44b49f053070c1e3d253, af3e13cfbd2f46d79292646cebbc89e7; no voters:
I20260808 22:28:24.504012 24806 raft_consensus.cc:2781] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P af3e13cfbd2f46d79292646cebbc89e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:28:24.504303 24806 raft_consensus.cc:686] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P af3e13cfbd2f46d79292646cebbc89e7 [term 1 LEADER]: Becoming Leader. State: Replica: af3e13cfbd2f46d79292646cebbc89e7, State: Running, Role: LEADER
I20260808 22:28:24.504666 24806 consensus_queue.cc:237] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P af3e13cfbd2f46d79292646cebbc89e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c4c33cc3b2b44b49f053070c1e3d253" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39213 } } peers { permanent_uuid: "b066d92a71184add8cfe4603f4ba82ff" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 43565 } } peers { permanent_uuid: "af3e13cfbd2f46d79292646cebbc89e7" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 33517 } }
I20260808 22:28:24.508431 24018 catalog_manager.cc:5495] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P af3e13cfbd2f46d79292646cebbc89e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to af3e13cfbd2f46d79292646cebbc89e7 (127.5.11.195). New cstate: current_term: 1 leader_uuid: "af3e13cfbd2f46d79292646cebbc89e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c4c33cc3b2b44b49f053070c1e3d253" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b066d92a71184add8cfe4603f4ba82ff" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 43565 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af3e13cfbd2f46d79292646cebbc89e7" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 33517 } health_report { overall_health: HEALTHY } } }
W20260808 22:28:24.528198 24556 tablet.cc:2369] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P b066d92a71184add8cfe4603f4ba82ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:28:24.570243 24791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:28:24.564231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59060 (local address 127.5.11.195:33517)
0808 22:28:24.564709 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0808 22:28:24.564716 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:28:24.564745 (+ 29us) server_negotiation.cc:407] Connection header received
0808 22:28:24.564841 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:28:24.564845 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:28:24.564912 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:28:24.565012 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:28:24.565711 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:28:24.566544 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:28:24.567989 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:28:24.568334 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:28:24.568438 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:28:24.568909 (+ 471us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:28:24.569488 (+ 579us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:28:24.569609 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0808 22:28:24.570031 (+ 422us) server_negotiation.cc:299] Negotiation successful
0808 22:28:24.570094 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372}
I20260808 22:28:24.577289 24487 raft_consensus.cc:1261] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P b066d92a71184add8cfe4603f4ba82ff [term 1 FOLLOWER]: Refusing update from remote peer af3e13cfbd2f46d79292646cebbc89e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:28:24.578056 24806 consensus_queue.cc:1035] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P af3e13cfbd2f46d79292646cebbc89e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b066d92a71184add8cfe4603f4ba82ff" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 43565 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:28:24.583920 24274 raft_consensus.cc:1261] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P 7c4c33cc3b2b44b49f053070c1e3d253 [term 1 FOLLOWER]: Refusing update from remote peer af3e13cfbd2f46d79292646cebbc89e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:28:24.584580 24806 consensus_queue.cc:1035] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P af3e13cfbd2f46d79292646cebbc89e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c4c33cc3b2b44b49f053070c1e3d253" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 39213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:28:24.595988 24816 mvcc.cc:205] Tried to move back new op lower bound from 7316390316340658176 to 7316390316054396928. Current Snapshot: MvccSnapshot[applied={T|T < 7316390316340658176}]
I20260808 22:28:24.607168 24820 mvcc.cc:205] Tried to move back new op lower bound from 7316390316340658176 to 7316390316054396928. Current Snapshot: MvccSnapshot[applied={T|T < 7316390316340658176}]
I20260808 22:28:24.610960 24018 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41506:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:28:24.611210 24018 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41506:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:28:24.614279 24018 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a8b8d4ca74154845a2162cc9a04d241b: Sending DeleteTablet for 3 replicas of tablet 23dfa8e2b1e2409f95ab0f7c41ea4896
I20260808 22:28:24.615165 24241 tablet_service.cc:1514] Processing DeleteTablet for tablet 23dfa8e2b1e2409f95ab0f7c41ea4896 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:28:24 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:43924
I20260808 22:28:24.615218 24659 tablet_service.cc:1514] Processing DeleteTablet for tablet 23dfa8e2b1e2409f95ab0f7c41ea4896 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:28:24 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:59048
I20260808 22:28:24.615702 24451 tablet_service.cc:1514] Processing DeleteTablet for tablet 23dfa8e2b1e2409f95ab0f7c41ea4896 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:28:24 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:43286
I20260808 22:28:24.616127 24827 tablet_replica.cc:330] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P af3e13cfbd2f46d79292646cebbc89e7: stopping tablet replica
I20260808 22:28:24.616465 24827 raft_consensus.cc:2229] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P af3e13cfbd2f46d79292646cebbc89e7 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:28:24.616503 24828 tablet_replica.cc:330] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P b066d92a71184add8cfe4603f4ba82ff: stopping tablet replica
I20260808 22:28:24.616824 24827 raft_consensus.cc:2258] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P af3e13cfbd2f46d79292646cebbc89e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:28:24.616914 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 24139
I20260808 22:28:24.616964 24828 raft_consensus.cc:2229] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P b066d92a71184add8cfe4603f4ba82ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:28:24.618122 24828 raft_consensus.cc:2258] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P b066d92a71184add8cfe4603f4ba82ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:28:24.618628 24827 ts_tablet_manager.cc:1905] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P af3e13cfbd2f46d79292646cebbc89e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:28:24.621800 24828 ts_tablet_manager.cc:1905] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P b066d92a71184add8cfe4603f4ba82ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:28:24.622345 24827 ts_tablet_manager.cc:1918] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P af3e13cfbd2f46d79292646cebbc89e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:28:24.622440 24827 log.cc:1196] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P af3e13cfbd2f46d79292646cebbc89e7: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-2/wal/wals/23dfa8e2b1e2409f95ab0f7c41ea4896
I20260808 22:28:24.622766 24827 ts_tablet_manager.cc:1939] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P af3e13cfbd2f46d79292646cebbc89e7: Deleting consensus metadata
I20260808 22:28:24.624017 24000 catalog_manager.cc:4841] TS af3e13cfbd2f46d79292646cebbc89e7 (127.5.11.195:33517): tablet 23dfa8e2b1e2409f95ab0f7c41ea4896 (table test-table [id=2f2fbcda1ede4c61b5fb5b88b297eaa9]) successfully deleted
I20260808 22:28:24.624781 24828 ts_tablet_manager.cc:1918] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P b066d92a71184add8cfe4603f4ba82ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:28:24.624876 24828 log.cc:1196] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P b066d92a71184add8cfe4603f4ba82ff: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786227991988852-5167-0/minicluster-data/ts-1/wal/wals/23dfa8e2b1e2409f95ab0f7c41ea4896
I20260808 22:28:24.625204 24828 ts_tablet_manager.cc:1939] T 23dfa8e2b1e2409f95ab0f7c41ea4896 P b066d92a71184add8cfe4603f4ba82ff: Deleting consensus metadata
I20260808 22:28:24.626477 24001 catalog_manager.cc:4841] TS b066d92a71184add8cfe4603f4ba82ff (127.5.11.194:43565): tablet 23dfa8e2b1e2409f95ab0f7c41ea4896 (table test-table [id=2f2fbcda1ede4c61b5fb5b88b297eaa9]) successfully deleted
W20260808 22:28:24.630781 24000 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.11.193:39213 (error 108)
I20260808 22:28:24.631125 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 24360
I20260808 22:28:24.645597 24787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:28:24.645143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:39213 (local address 127.0.0.1:43932)
0808 22:28:24.645419 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0808 22:28:24.645497 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.193:39213: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172}
W20260808 22:28:24.645823 24000 catalog_manager.cc:4584] TS 7c4c33cc3b2b44b49f053070c1e3d253 (127.5.11.193:39213): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 23dfa8e2b1e2409f95ab0f7c41ea4896: Network error: Client connection negotiation failed: client connection to 127.5.11.193:39213: connect: Connection refused (error 111)
I20260808 22:28:24.646724 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 24560
I20260808 22:28:24.660490 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 23977
I20260808 22:28:24.688251 5167 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 22:28:25.032538 5167 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 22:28:25.032691 5167 mini_ranger.cc:66] Stopping Ranger...
I20260808 22:28:25.377481 5167 mini_ranger.cc:68] Stopped Ranger
2026-08-08 22:28:25.377 UTC [23339] LOG: received smart shutdown request
2026-08-08 22:28:25.381 UTC [23339] LOG: background worker "logical replication launcher" (PID 23350) exited with exit code 1
2026-08-08 22:28:25.382 UTC [23343] LOG: shutting down
2026-08-08 22:28:25.543 UTC [23339] LOG: database system is shut down
2026-08-08T22:28:25Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (46967 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:28:25 dist-test-slave-7f01 krb5kdc[24846](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:28:25 dist-test-slave-7f01 krb5kdc[24846](info): set up 2 sockets
Aug 08 22:28:25 dist-test-slave-7f01 krb5kdc[24846](info): commencing operation
krb5kdc: starting...
W20260808 22:28:26.956193 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.268s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:28:27 dist-test-slave-7f01 krb5kdc[24846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228107, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:28:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:28:27Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.5.11.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.5.11.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.5.11.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/rangeradmin_127.5.11.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.5.11.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/rangeradmin_127.5.11.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.5.11.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.5.11.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.5.11.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/rangerlookup_127.5.11.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.5.11.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/rangerlookup_127.5.11.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.11.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.5.11.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.11.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/HTTP_127.5.11.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.11.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/HTTP_127.5.11.212@KRBTEST.COM.keytab.
I20260808 22:28:27.130767 5167 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskM4Eo1i/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/postgres -l logfile start
2026-08-08 22:28:29.552 UTC [24898] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-08 22:28:29.552 UTC [24898] LOG: listening on IPv4 address "127.5.11.212", port 53535
2026-08-08 22:28:29.555 UTC [24898] LOG: listening on Unix socket "/tmp/.s.PGSQL.53535"
2026-08-08 22:28:29.577 UTC [24902] LOG: database system was shut down at 2026-08-08 22:28:28 UTC
2026-08-08 22:28:29.581 UTC [24898] LOG: database system is ready to accept connections
I20260808 22:28:30.762754 5167 mini_postgres.cc:96] Postgres bound to 53535
2026-08-08 22:28:30.770 UTC [24914] FATAL: database "slave" does not exist
127.5.11.212:53535 - accepting connections
I20260808 22:28:30.771186 5167 mini_ranger.cc:161] Starting Ranger...
I20260808 22:28:30.787086 5167 mini_ranger.cc:84] Created miniranger Postgres user
I20260808 22:28:31.251492 5167 mini_ranger.cc:87] Created ranger Postgres database
I20260808 22:28:31.251610 5167 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ranger-admin
2026-08-08 22:28:31,535 [I] DB FLAVOR :POSTGRES
2026-08-08 22:28:31,535 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 22:28:31,535 [I] Checking connection..
2026-08-08 22:28:31,536 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:53535/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-08 22:28:31,769 [I] Checking connection passed.
2026-08-08 22:28:31,770 [I] --------- Verifying version history table ---------
2026-08-08 22:28:31,770 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:53535/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 22:28:32,033 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 22:28:32,034 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:53535/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 22:28:32,267 [I] Table x_db_version_h does not exist in database ranger
2026-08-08 22:28:32,267 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-08 22:28:32,267 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:53535/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-08 22:28:32.528 UTC [25025] WARNING: there is no transaction in progress
2026-08-08 22:28:32,534 [I] create_dbversion_catalog.sql file imported successfully
2026-08-08 22:28:32,535 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:53535/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-08 22:28:32,787 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-08 22:28:32,787 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 22:28:32,787 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:53535/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-08 22:28:33,047 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:53535/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-08 22:28:33,278 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:53535/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-7f01.c.gcp-upstream.internal','N') ;"
2026-08-08 22:28:33,501 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:53535/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 22:28:33,765 [I] Table x_portal_user does not exist in database ranger
2026-08-08 22:28:33,766 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:53535/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 22:28:34,023 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-08 22:28:34,023 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:53535/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 22:28:34,260 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:53535/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 22:28:34,507 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-08 22:28:34,507 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:53535/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-08 22:28:35.452 UTC [25199] WARNING: there is no transaction in progress
2026-08-08 22:28:35.468 UTC [25199] WARNING: there is no transaction in progress
2026-08-08 22:28:35.484 UTC [25199] WARNING: there is no transaction in progress
2026-08-08 22:28:35.499 UTC [25199] WARNING: there is no transaction in progress
2026-08-08 22:28:35.514 UTC [25199] WARNING: there is no transaction in progress
2026-08-08 22:28:35.531 UTC [25199] WARNING: there is no transaction in progress
2026-08-08 22:28:35.582 UTC [25199] WARNING: there is no transaction in progress
2026-08-08 22:28:35.588 UTC [25199] WARNING: there is no transaction in progress
2026-08-08 22:28:35.602 UTC [25199] WARNING: there is no transaction in progress
2026-08-08 22:28:35.612 UTC [25199] WARNING: there is no transaction in progress
2026-08-08 22:28:35.621 UTC [25199] WARNING: there is no transaction in progress
2026-08-08 22:28:35.635 UTC [25199] WARNING: there is no transaction in progress
2026-08-08 22:28:35.644 UTC [25199] WARNING: there is no transaction in progress
2026-08-08 22:28:35.650 UTC [25199] WARNING: there is no transaction in progress
2026-08-08 22:28:35.656 UTC [25199] WARNING: there is no transaction in progress
2026-08-08 22:28:35.663 UTC [25199] WARNING: there is no transaction in progress
2026-08-08 22:28:36,288 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-08 22:28:36,289 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:53535/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-08 22:28:36,544 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-08 22:28:36,545 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:53535/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-08 22:28:36,798 [I] Table x_portal_user already exists in database 'ranger'
2026-08-08 22:28:36,799 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:53535/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-08 22:28:37,063 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-08 22:28:37,063 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:53535/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-08 22:28:37,313 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:53535/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-08 22:28:37,575 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:53535/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-7f01.c.gcp-upstream.internal';"
2026-08-08 22:28:37,797 [I] CORE_DB_SCHEMA import status has been updated
2026-08-08 22:28:37,797 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:53535/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-08 22:28:38,031 [I] DB_PATCHES have already been applied
I20260808 22:28:38.037648 5167 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ranger-admin:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskM4Eo1i/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260808 22:28:38.037720 5167 mini_ranger.cc:193] Using host: 127.5.11.212
I20260808 22:28:38.042538 5167 mini_ranger.cc:227] Ranger admin URL: http://127.5.11.212:42213
Aug 08, 2026 10:28:38 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 10:28:38 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 10:28:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ranger-admin/ews/webapp
Aug 08, 2026 10:28:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 08, 2026 10:28:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 08, 2026 10:28:39 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ranger-admin/ews/webapp].
Aug 08, 2026 10:28:39 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260808 22:28:39.431026 5167 mini_ranger.cc:160] Time spent starting Ranger: real 8.660s user 0.000s sys 0.012s
Aug 08, 2026 10:28:59 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 08, 2026 10:28:59 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 08, 2026 10:28:59 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 08, 2026 10:28:59 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 08, 2026 10:28:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 08, 2026 10:28:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 08, 2026 10:28:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 08, 2026 10:28:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 08, 2026 10:28:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 08, 2026 10:28:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 08, 2026 10:28:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 08, 2026 10:28:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 08, 2026 10:28:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 08, 2026 10:28:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 08, 2026 10:28:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 08, 2026 10:28:59 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 08, 2026 10:28:59 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 08, 2026 10:29:00 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260808 22:29:00.999369 5167 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.5.11.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.5.11.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.5.11.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/rangerkms_127.5.11.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.5.11.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/rangerkms_127.5.11.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.5.11.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.5.11.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.5.11.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/HTTP_127.5.11.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.5.11.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/HTTP_127.5.11.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 08 22:29:01 dist-test-slave-7f01 krb5kdc[24846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228141, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260808 22:29:01.086074 5167 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260808 22:29:01.096361 5167 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260808 22:29:01.949623 5167 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260808 22:29:01.949728 5167 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ranger-kms
I20260808 22:29:01.949747 5167 mini_ranger_kms.cc:227] Using postgres at 127.5.11.212:53535
2026-08-08 22:29:02,130 [I] DB FLAVOR :POSTGRES
2026-08-08 22:29:02,130 [I] --------- Verifying Ranger DB connection ---------
2026-08-08 22:29:02,130 [I] Checking connection
2026-08-08 22:29:02,130 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:53535/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-08 22:29:02,377 [I] connection success
2026-08-08 22:29:02,377 [I] --------- Verifying Ranger DB tables ---------
2026-08-08 22:29:02,377 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-08 22:29:02,377 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:53535/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-08 22:29:02,610 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-08 22:29:02,610 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-08 22:29:02,611 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-08 22:29:02,611 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.5.11.212:53535/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-08 22:29:02,883 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260808 22:29:03.362510 5167 mini_ranger_kms.cc:310] Created kms service
I20260808 22:29:03.546619 5167 mini_ranger_kms.cc:326] Created kudu user
I20260808 22:29:03.590816 5167 mini_ranger_kms.cc:343] Created rangerkms user
Aug 08 22:29:03 dist-test-slave-7f01 krb5kdc[24846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228141, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.5.11.212@KRBTEST.COM
I20260808 22:29:03.998071 5167 mini_ranger_kms.cc:384] Added ranger policy
I20260808 22:29:03.998261 5167 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ranger-kms:/tmp/dist-test-taskM4Eo1i/build/debug/bin/postgresql.jar:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskM4Eo1i/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260808 22:29:03.998281 5167 mini_ranger_kms.cc:240] Using host: 127.5.11.212
I20260808 22:29:04.003119 5167 mini_ranger_kms.cc:276] Ranger KMS PID: 25498
I20260808 22:29:04.003204 5167 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.5.11.212:34867
Aug 08, 2026 10:29:04 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 08, 2026 10:29:04 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 08, 2026 10:29:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 08, 2026 10:29:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 08, 2026 10:29:05 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ranger-kms/ews/webapp].
Aug 08, 2026 10:29:05 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 08 22:29:06 dist-test-slave-7f01 krb5kdc[24846](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786228146, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.11.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 08 22:29:06 dist-test-slave-7f01 krb5kdc[24846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228146, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.11.212@KRBTEST.COM for HTTP/127.5.11.212@KRBTEST.COM
I20260808 22:29:06.820436 5167 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.734s user 0.006s sys 0.159s
I20260808 22:29:06.820567 5167 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260808 22:29:06.820649 5167 mini_ranger_kms.cc:401] 127.5.11.212:34867/kms/v1/keys
Aug 08 22:29:06 dist-test-slave-7f01 krb5kdc[24846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228146, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.5.11.212@KRBTEST.COM for HTTP/127.5.11.212@KRBTEST.COM
Aug 08 22:29:09 dist-test-slave-7f01 krb5kdc[24846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.5.11.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
I20260808 22:29:09.062917 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:38533
--webserver_interface=127.5.11.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:37937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:38533
--ranger_config_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.11.212:34867/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:29:09.177603 25568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:29:09.177834 25568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:29:09.177871 25568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:29:09.182348 25568 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:29:09.182407 25568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:29:09.182426 25568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:29:09.182464 25568 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:29:09.182494 25568 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:29:09.187690 25568 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:37937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/master-0/wal
--ranger_kms_url=127.5.11.212:34867/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:38533
--ranger_config_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:38533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:29:09.188699 25568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:29:09.189733 25568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:29:09.197906 25577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:09.198101 25579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:09.198107 25575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:29:09.198233 25568 server_base.cc:1034] running on GCE node
I20260808 22:29:09.198832 25568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:29:09.199930 25568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:29:09.201105 25568 hybrid_clock.cc:648] HybridClock initialized: now 1786228149201087 us; error 28 us; skew 500 ppm
Aug 08 22:29:09 dist-test-slave-7f01 krb5kdc[24846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228149, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:29:09.208107 25568 init.cc:377] Logged in from keytab as kudu/127.5.11.254@KRBTEST.COM (short username kudu)
I20260808 22:29:09.209003 25568 webserver.cc:460] Webserver started at http://127.5.11.254:45193/ using document root <none> and password file <none>
I20260808 22:29:09.209520 25568 fs_manager.cc:359] Metadata directory not provided
I20260808 22:29:09.209578 25568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:29:09.209754 25568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 22:29:09 dist-test-slave-7f01 krb5kdc[24846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228149, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for HTTP/127.5.11.212@KRBTEST.COM
I20260808 22:29:09.260653 25568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "a8ebf5ce638449f8a562ef695010caf5"
format_stamp: "Formatted at 2026-08-08 22:29:09 on dist-test-slave-7f01"
server_key: "2c109f24e30eb07c8752f900187360f8"
server_key_iv: "0d79df9b29d779cf78978e86684c2b38"
server_key_version: "kuduclusterkey@0"
I20260808 22:29:09.261241 25568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "a8ebf5ce638449f8a562ef695010caf5"
format_stamp: "Formatted at 2026-08-08 22:29:09 on dist-test-slave-7f01"
server_key: "2c109f24e30eb07c8752f900187360f8"
server_key_iv: "0d79df9b29d779cf78978e86684c2b38"
server_key_version: "kuduclusterkey@0"
I20260808 22:29:09.265107 25568 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 22:29:09.278510 25589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:09.280138 25568 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:29:09.280278 25568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "a8ebf5ce638449f8a562ef695010caf5"
format_stamp: "Formatted at 2026-08-08 22:29:09 on dist-test-slave-7f01"
server_key: "2c109f24e30eb07c8752f900187360f8"
server_key_iv: "0d79df9b29d779cf78978e86684c2b38"
server_key_version: "kuduclusterkey@0"
I20260808 22:29:09.280406 25568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:29:09.299962 25568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:29:09.302908 25568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:29:09.303224 25568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:29:09.310555 25568 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:38533
I20260808 22:29:09.310580 25677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:38533 every 8 connection(s)
I20260808 22:29:09.311491 25568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:29:09.314291 25679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:29:09.319824 25679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5: Bootstrap starting.
I20260808 22:29:09.320623 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 25568
I20260808 22:29:09.320775 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:29:09.321053 5167 external_mini_cluster.cc:1428] Setting key 063ab50ec9249a56ad78d32a32594ad2
I20260808 22:29:09.322561 25679 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5: Neither blocks nor log segments found. Creating new log.
I20260808 22:29:09.323290 25679 log.cc:824] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5: Log is configured to *not* fsync() on all Append() calls
I20260808 22:29:09.325537 25679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5: No bootstrap required, opened a new log
Aug 08 22:29:09 dist-test-slave-7f01 krb5kdc[24846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:29:09.330103 25679 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ebf5ce638449f8a562ef695010caf5" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 38533 } }
I20260808 22:29:09.330334 25679 raft_consensus.cc:374] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:29:09.330400 25679 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ebf5ce638449f8a562ef695010caf5, State: Initialized, Role: FOLLOWER
I20260808 22:29:09.330816 25679 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ebf5ce638449f8a562ef695010caf5" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 38533 } }
I20260808 22:29:09.330971 25679 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:29:09.331096 25679 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:29:09.331230 25679 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:29:09.332374 25679 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ebf5ce638449f8a562ef695010caf5" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 38533 } }
I20260808 22:29:09.332752 25679 leader_election.cc:302] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8ebf5ce638449f8a562ef695010caf5; no voters:
I20260808 22:29:09.333142 25679 leader_election.cc:288] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:29:09.333231 25685 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:29:09.333461 25685 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5 [term 1 LEADER]: Becoming Leader. State: Replica: a8ebf5ce638449f8a562ef695010caf5, State: Running, Role: LEADER
I20260808 22:29:09.333792 25685 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ebf5ce638449f8a562ef695010caf5" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 38533 } }
I20260808 22:29:09.334468 25679 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:29:09.335467 25687 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a8ebf5ce638449f8a562ef695010caf5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ebf5ce638449f8a562ef695010caf5" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 38533 } } }
I20260808 22:29:09.335590 25687 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5 [sys.catalog]: This master's current role is: LEADER
I20260808 22:29:09.335839 25682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:09.322164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59048 (local address 127.5.11.254:38533)
0808 22:29:09.322597 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:09.322612 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:09.322646 (+ 34us) server_negotiation.cc:407] Connection header received
0808 22:29:09.323276 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:09.323292 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:09.323663 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:09.324078 (+ 415us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:09.324944 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:09.325946 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:09.327170 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:09.327606 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:09.327719 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:09.330437 (+ 2718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:09.330445 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:09.330474 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:09.330510 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:09.332554 (+ 2044us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:09.332998 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:09.333006 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:09.333011 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:09.333165 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:09.333551 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:09.333554 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:09.333556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:09.334021 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:09.334229 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:09.334515 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 22:29:09.334791 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":142,"threads_started":1}
I20260808 22:29:09.336007 25688 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8ebf5ce638449f8a562ef695010caf5. Latest consensus state: current_term: 1 leader_uuid: "a8ebf5ce638449f8a562ef695010caf5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ebf5ce638449f8a562ef695010caf5" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 38533 } } }
I20260808 22:29:09.336288 25688 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5 [sys.catalog]: This master's current role is: LEADER
I20260808 22:29:09.337178 25679 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 22:29:09.338282 25699 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:29:09.344699 25699 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:29:09.350781 25699 catalog_manager.cc:1347] Generated new cluster ID: 3ca0e85d464c46668a7d7942fe761356
I20260808 22:29:09.350888 25699 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:29:09.370747 25699 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:29:09.371840 25699 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:29:09.377601 25699 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5: Generated new TSK 0
I20260808 22:29:09.378490 25699 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.11.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab.
Entry for principal kudu/127.5.11.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab.
I20260808 22:29:10.071179 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.11.212:34867/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.193:0
--local_ip_for_outbound_sockets=127.5.11.193
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:38533
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:37937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:29:10.217120 25744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:29:10.217383 25744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:29:10.217453 25744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:29:10.223695 25744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:29:10.223845 25744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:29:10.223959 25744 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.193
I20260808 22:29:10.231535 25744 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:37937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.5.11.212:34867/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:38533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.193
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:29:10.233009 25744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:29:10.234431 25744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:29:10.250505 25756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:10.250978 25757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:10.251120 25761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:29:10.252976 25744 server_base.cc:1034] running on GCE node
I20260808 22:29:10.253571 25744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:29:10.254350 25744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:29:10.261765 25744 hybrid_clock.cc:648] HybridClock initialized: now 1786228150261719 us; error 43 us; skew 500 ppm
Aug 08 22:29:10 dist-test-slave-7f01 krb5kdc[24846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228150, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:29:10.265739 25744 init.cc:377] Logged in from keytab as kudu/127.5.11.193@KRBTEST.COM (short username kudu)
I20260808 22:29:10.266911 25744 webserver.cc:460] Webserver started at http://127.5.11.193:39447/ using document root <none> and password file <none>
I20260808 22:29:10.267834 25744 fs_manager.cc:359] Metadata directory not provided
I20260808 22:29:10.267992 25744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:29:10.268322 25744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 22:29:10 dist-test-slave-7f01 krb5kdc[24846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228150, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for HTTP/127.5.11.212@KRBTEST.COM
I20260808 22:29:10.304360 25744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-0/data/instance:
uuid: "a6817d6abcac41469590f9b7d73710f2"
format_stamp: "Formatted at 2026-08-08 22:29:10 on dist-test-slave-7f01"
server_key: "189d5395d596b78a54374a5f031aa1af"
server_key_iv: "ed2b139b1e9e0b7de1dca7290711637b"
server_key_version: "kuduclusterkey@0"
I20260808 22:29:10.307950 25744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance:
uuid: "a6817d6abcac41469590f9b7d73710f2"
format_stamp: "Formatted at 2026-08-08 22:29:10 on dist-test-slave-7f01"
server_key: "189d5395d596b78a54374a5f031aa1af"
server_key_iv: "ed2b139b1e9e0b7de1dca7290711637b"
server_key_version: "kuduclusterkey@0"
I20260808 22:29:10.313294 25744 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 22:29:10.355175 25772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:10.357126 25744 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.002s sys 0.000s
I20260808 22:29:10.357309 25744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-0/wal
uuid: "a6817d6abcac41469590f9b7d73710f2"
format_stamp: "Formatted at 2026-08-08 22:29:10 on dist-test-slave-7f01"
server_key: "189d5395d596b78a54374a5f031aa1af"
server_key_iv: "ed2b139b1e9e0b7de1dca7290711637b"
server_key_version: "kuduclusterkey@0"
I20260808 22:29:10.357479 25744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:29:10.380185 25744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:29:10.383585 25744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:29:10.383821 25744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:29:10.384495 25744 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:29:10.385600 25744 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:29:10.385670 25744 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:10.385759 25744 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:29:10.385795 25744 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:10.397745 25744 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.193:38029
I20260808 22:29:10.397775 25968 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.193:38029 every 8 connection(s)
I20260808 22:29:10.398921 25744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
I20260808 22:29:10.405884 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 25744
I20260808 22:29:10.406025 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance
I20260808 22:29:10.406314 5167 external_mini_cluster.cc:1428] Setting key 32b779bfffbc9da07e1d607529308b85
Aug 08 22:29:10 dist-test-slave-7f01 krb5kdc[24846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228150, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
Aug 08 22:29:10 dist-test-slave-7f01 krb5kdc[24846](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786228150, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:29:10.419186 25973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:10.401517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:38533 (local address 127.5.11.193:57745)
0808 22:29:10.402020 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:10.402075 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:10.404410 (+ 2335us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:10.404984 (+ 574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:10.404991 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:10.405264 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:29:10.407058 (+ 1794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:10.407081 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:10.408320 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:10.408323 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:10.409734 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:10.409747 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:10.410519 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:10.410523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:10.410734 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:10.411457 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:29:10.411479 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:29:10.413279 (+ 1800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:29:10.415640 (+ 2361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:10.415647 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:10.415664 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:10.416069 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:10.416431 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:10.416434 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:10.416436 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:10.416618 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:10.417165 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:29:10.417171 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:29:10.417502 (+ 331us) client_negotiation.cc:769] Sending connection context
0808 22:29:10.417713 (+ 211us) client_negotiation.cc:240] Negotiation successful
0808 22:29:10.418071 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":147,"threads_started":1}
I20260808 22:29:10.419996 25972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:10.401065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:57745 (local address 127.5.11.254:38533)
0808 22:29:10.404282 (+ 3217us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:10.404287 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:10.404308 (+ 21us) server_negotiation.cc:407] Connection header received
0808 22:29:10.404624 (+ 316us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:10.404627 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:10.404690 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:10.404778 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:10.407271 (+ 2493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:10.408161 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:10.409884 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:10.410224 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:10.410325 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:10.413488 (+ 3163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:10.413492 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:10.413495 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:10.413524 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:10.415516 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:10.416206 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:10.416211 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:10.416213 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:10.416315 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:10.416744 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:10.416747 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:10.416749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:10.417005 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:10.417106 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:10.419642 (+ 2536us) server_negotiation.cc:299] Negotiation successful
0808 22:29:10.419806 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3133,"thread_start_us":64,"threads_started":1}
WARNING: no policy specified for kudu/127.5.11.194@KRBTEST.COM; defaulting to no policy
I20260808 22:29:10.421746 25969 heartbeater.cc:347] Connected to a master server at 127.5.11.254:38533
I20260808 22:29:10.422013 25969 heartbeater.cc:464] Registering TS with master...
I20260808 22:29:10.422609 25969 heartbeater.cc:511] Master 127.5.11.254:38533 requested a full tablet report, sending...
I20260808 22:29:10.424463 25617 ts_manager.cc:194] Registered new tserver with Master: a6817d6abcac41469590f9b7d73710f2 (127.5.11.193:38029)
I20260808 22:29:10.426024 25617 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.193@KRBTEST.COM'} at 127.5.11.193:57745
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab.
Entry for principal kudu/127.5.11.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab.
I20260808 22:29:10.455582 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.11.212:34867/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.194:0
--local_ip_for_outbound_sockets=127.5.11.194
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:38533
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:37937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:29:10.598489 25978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:29:10.598765 25978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:29:10.598896 25978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:29:10.603008 25978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:29:10.603082 25978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:29:10.603178 25978 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.194
I20260808 22:29:10.608397 25978 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:37937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.5.11.212:34867/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:38533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.194
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:29:10.609460 25978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:29:10.610489 25978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:29:10.622509 25991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:10.622535 25997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:10.623258 25993 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:29:10.623565 25978 server_base.cc:1034] running on GCE node
I20260808 22:29:10.624043 25978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:29:10.624784 25978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:29:10.632242 25978 hybrid_clock.cc:648] HybridClock initialized: now 1786228150632156 us; error 96 us; skew 500 ppm
Aug 08 22:29:10 dist-test-slave-7f01 krb5kdc[24846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228150, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:29:10.635573 25978 init.cc:377] Logged in from keytab as kudu/127.5.11.194@KRBTEST.COM (short username kudu)
I20260808 22:29:10.636670 25978 webserver.cc:460] Webserver started at http://127.5.11.194:43009/ using document root <none> and password file <none>
I20260808 22:29:10.637274 25978 fs_manager.cc:359] Metadata directory not provided
I20260808 22:29:10.637341 25978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:29:10.637596 25978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 22:29:10 dist-test-slave-7f01 krb5kdc[24846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228150, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for HTTP/127.5.11.212@KRBTEST.COM
I20260808 22:29:10.656525 25978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-1/data/instance:
uuid: "a0a9fc9ff93043b39554e5048156bfd3"
format_stamp: "Formatted at 2026-08-08 22:29:10 on dist-test-slave-7f01"
server_key: "fcf315238c750fa1e1d8e7acd5de7f22"
server_key_iv: "21cfdd372f70d4b2496526894228ee3d"
server_key_version: "kuduclusterkey@0"
I20260808 22:29:10.657395 25978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance:
uuid: "a0a9fc9ff93043b39554e5048156bfd3"
format_stamp: "Formatted at 2026-08-08 22:29:10 on dist-test-slave-7f01"
server_key: "fcf315238c750fa1e1d8e7acd5de7f22"
server_key_iv: "21cfdd372f70d4b2496526894228ee3d"
server_key_version: "kuduclusterkey@0"
I20260808 22:29:10.663152 25978 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260808 22:29:10.688146 26007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:10.690009 25978 fs_manager.cc:725] Time spent opening block manager: real 0.009s user 0.002s sys 0.000s
I20260808 22:29:10.690224 25978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-1/wal
uuid: "a0a9fc9ff93043b39554e5048156bfd3"
format_stamp: "Formatted at 2026-08-08 22:29:10 on dist-test-slave-7f01"
server_key: "fcf315238c750fa1e1d8e7acd5de7f22"
server_key_iv: "21cfdd372f70d4b2496526894228ee3d"
server_key_version: "kuduclusterkey@0"
I20260808 22:29:10.690387 25978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:29:10.730427 25978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:29:10.734211 25978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:29:10.734477 25978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:29:10.735248 25978 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:29:10.736351 25978 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:29:10.736452 25978 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:10.736577 25978 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:29:10.736641 25978 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:10.749087 25978 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.194:46345
I20260808 22:29:10.751488 25978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
I20260808 22:29:10.754731 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 25978
I20260808 22:29:10.754878 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance
I20260808 22:29:10.755182 5167 external_mini_cluster.cc:1428] Setting key d6d93f09a65f258bcbf2cd86fff45508
I20260808 22:29:10.753194 26197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.194:46345 every 8 connection(s)
Aug 08 22:29:10 dist-test-slave-7f01 krb5kdc[24846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228150, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:29:10.788185 25972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:10.756499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:44247 (local address 127.5.11.254:38533)
0808 22:29:10.756653 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:10.756658 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:10.760193 (+ 3535us) server_negotiation.cc:407] Connection header received
0808 22:29:10.761154 (+ 961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:10.761159 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:10.761217 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:10.761294 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:10.764765 (+ 3471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:10.765580 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:10.769730 (+ 4150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:10.770083 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:10.770165 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:10.775304 (+ 5139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:10.775308 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:10.775311 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:10.775339 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:10.777453 (+ 2114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:10.784377 (+ 6924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:10.784382 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:10.784384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:10.784503 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:10.785159 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:10.785162 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:10.785165 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:10.785420 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:10.785514 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:10.787833 (+ 2319us) server_negotiation.cc:299] Negotiation successful
0808 22:29:10.787984 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 22:29:10.791391 26201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:10.755385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:38533 (local address 127.5.11.194:44247)
0808 22:29:10.760040 (+ 4655us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:10.760085 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:10.760893 (+ 808us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:10.761612 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:10.761621 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:10.761994 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:29:10.764064 (+ 2070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:10.764087 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:10.766700 (+ 2613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:10.766703 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:10.769573 (+ 2870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:10.769586 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:10.770674 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:10.770679 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:10.770880 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:10.771601 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:29:10.771624 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:29:10.775101 (+ 3477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:29:10.777598 (+ 2497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:10.777604 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:10.777620 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:10.784239 (+ 6619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:10.784633 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:10.784636 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:10.784639 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:10.785036 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:10.785590 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:29:10.785597 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:29:10.786185 (+ 588us) client_negotiation.cc:769] Sending connection context
0808 22:29:10.788737 (+ 2552us) client_negotiation.cc:240] Negotiation successful
0808 22:29:10.789101 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4407,"thread_start_us":137,"threads_started":1}
WARNING: no policy specified for kudu/127.5.11.195@KRBTEST.COM; defaulting to no policy
I20260808 22:29:10.800258 26198 heartbeater.cc:347] Connected to a master server at 127.5.11.254:38533
I20260808 22:29:10.800539 26198 heartbeater.cc:464] Registering TS with master...
I20260808 22:29:10.801151 26198 heartbeater.cc:511] Master 127.5.11.254:38533 requested a full tablet report, sending...
I20260808 22:29:10.802374 25617 ts_manager.cc:194] Registered new tserver with Master: a0a9fc9ff93043b39554e5048156bfd3 (127.5.11.194:46345)
I20260808 22:29:10.803153 25617 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.194@KRBTEST.COM'} at 127.5.11.194:44247
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab.
Entry for principal kudu/127.5.11.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab.
I20260808 22:29:10.842770 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.11.212:34867/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.195:0
--local_ip_for_outbound_sockets=127.5.11.195
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:38533
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:37937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 22:29:10 dist-test-slave-7f01 krb5kdc[24846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228150, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for HTTP/127.5.11.212@KRBTEST.COM
W20260808 22:29:11.061347 26206 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:29:11.061753 26206 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:29:11.061908 26206 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:29:11.068388 26206 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:29:11.068466 26206 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:29:11.068557 26206 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.195
I20260808 22:29:11.077045 26206 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:37937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.5.11.212:34867/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:38533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26206
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.195
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:29:11.078495 26206 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:29:11.079941 26206 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:29:11.094084 26222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:11.094085 26216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:11.094306 26213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:29:11.094803 26206 server_base.cc:1034] running on GCE node
I20260808 22:29:11.095206 26206 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:29:11.096056 26206 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:29:11.097239 26206 hybrid_clock.cc:648] HybridClock initialized: now 1786228151097205 us; error 47 us; skew 500 ppm
Aug 08 22:29:11 dist-test-slave-7f01 krb5kdc[24846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228151, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:29:11.100111 26206 init.cc:377] Logged in from keytab as kudu/127.5.11.195@KRBTEST.COM (short username kudu)
I20260808 22:29:11.100950 26206 webserver.cc:460] Webserver started at http://127.5.11.195:35219/ using document root <none> and password file <none>
I20260808 22:29:11.101529 26206 fs_manager.cc:359] Metadata directory not provided
I20260808 22:29:11.101616 26206 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:29:11.101859 26206 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 08 22:29:11 dist-test-slave-7f01 krb5kdc[24846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228151, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for HTTP/127.5.11.212@KRBTEST.COM
I20260808 22:29:11.120138 26206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-2/data/instance:
uuid: "0ebf78ad79c44a4f82b68aa49e9f1ac0"
format_stamp: "Formatted at 2026-08-08 22:29:11 on dist-test-slave-7f01"
server_key: "bc67cb4120b59d5ea0db97f616184dd4"
server_key_iv: "67eaf319254a56539afc0626b3ff7ec7"
server_key_version: "kuduclusterkey@0"
I20260808 22:29:11.120935 26206 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance:
uuid: "0ebf78ad79c44a4f82b68aa49e9f1ac0"
format_stamp: "Formatted at 2026-08-08 22:29:11 on dist-test-slave-7f01"
server_key: "bc67cb4120b59d5ea0db97f616184dd4"
server_key_iv: "67eaf319254a56539afc0626b3ff7ec7"
server_key_version: "kuduclusterkey@0"
I20260808 22:29:11.136965 26206 fs_manager.cc:691] Time spent creating directory manager: real 0.016s user 0.005s sys 0.000s
I20260808 22:29:11.168167 26234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:11.170081 26206 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.002s sys 0.001s
I20260808 22:29:11.170295 26206 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-2/wal
uuid: "0ebf78ad79c44a4f82b68aa49e9f1ac0"
format_stamp: "Formatted at 2026-08-08 22:29:11 on dist-test-slave-7f01"
server_key: "bc67cb4120b59d5ea0db97f616184dd4"
server_key_iv: "67eaf319254a56539afc0626b3ff7ec7"
server_key_version: "kuduclusterkey@0"
I20260808 22:29:11.170538 26206 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:29:11.198881 26206 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:29:11.202461 26206 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:29:11.202683 26206 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:29:11.203411 26206 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:29:11.204475 26206 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:29:11.204577 26206 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:11.204672 26206 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:29:11.204733 26206 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:11.217773 26206 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.195:42989
I20260808 22:29:11.218963 26206 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
I20260808 22:29:11.219539 26407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.195:42989 every 8 connection(s)
I20260808 22:29:11.224481 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 26206
I20260808 22:29:11.224637 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance
I20260808 22:29:11.224992 5167 external_mini_cluster.cc:1428] Setting key 964de16b0a9fb7748af1bddc3c3267fe
Aug 08 22:29:11 dist-test-slave-7f01 krb5kdc[24846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228150, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for HTTP/127.5.11.212@KRBTEST.COM
Aug 08 22:29:11 dist-test-slave-7f01 krb5kdc[24846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228151, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:29:11.243080 25972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:11.223496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:54117 (local address 127.5.11.254:38533)
0808 22:29:11.223651 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:11.223656 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:11.224476 (+ 820us) server_negotiation.cc:407] Connection header received
0808 22:29:11.230858 (+ 6382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:11.230862 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:11.230920 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:11.230974 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:11.232750 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:11.233609 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:11.235261 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:11.235646 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:11.235817 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:11.238870 (+ 3053us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:11.238875 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:11.238878 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:11.238907 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:11.240778 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:11.241449 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:11.241452 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:11.241454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:11.241542 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:11.241929 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:11.241932 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:11.241934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:11.242139 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:11.242242 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:11.242767 (+ 525us) server_negotiation.cc:299] Negotiation successful
0808 22:29:11.242906 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 22:29:11.247339 26412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:11.222324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:38533 (local address 127.5.11.195:54117)
0808 22:29:11.224168 (+ 1844us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:11.224237 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:11.230332 (+ 6095us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:11.231415 (+ 1083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:11.231424 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:11.231892 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:29:11.232585 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:11.232603 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:11.233754 (+ 1151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:11.233758 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:11.235117 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:11.235128 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:11.236010 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:11.236015 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:11.236242 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:11.236888 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:29:11.236910 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:29:11.238670 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:29:11.240929 (+ 2259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:11.240935 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:11.240952 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:11.241317 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:11.241636 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:11.241640 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:11.241642 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:11.241804 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:11.242242 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:29:11.242251 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:29:11.242517 (+ 266us) client_negotiation.cc:769] Sending connection context
0808 22:29:11.242706 (+ 189us) client_negotiation.cc:240] Negotiation successful
0808 22:29:11.243026 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1560,"thread_start_us":144,"threads_started":1}
I20260808 22:29:11.248885 26408 heartbeater.cc:347] Connected to a master server at 127.5.11.254:38533
I20260808 22:29:11.249135 26408 heartbeater.cc:464] Registering TS with master...
I20260808 22:29:11.249730 26408 heartbeater.cc:511] Master 127.5.11.254:38533 requested a full tablet report, sending...
I20260808 22:29:11.251112 25617 ts_manager.cc:194] Registered new tserver with Master: 0ebf78ad79c44a4f82b68aa49e9f1ac0 (127.5.11.195:42989)
I20260808 22:29:11.251899 25617 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.195@KRBTEST.COM'} at 127.5.11.195:54117
I20260808 22:29:11.257081 5167 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:29:11.257912 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 25744
I20260808 22:29:11.280932 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 25978
I20260808 22:29:11.354790 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 26206
I20260808 22:29:11.372432 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 25568
I20260808 22:29:11.384938 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:38533
--webserver_interface=127.5.11.254
--webserver_port=45193
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:37937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:38533
--ranger_config_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.11.212:34867/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.11.212:34867/kms with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:29:11.499579 26415 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:29:11.499898 26415 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:29:11.499977 26415 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:29:11.503877 26415 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:29:11.503958 26415 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:29:11.503995 26415 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:29:11.504048 26415 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:29:11.504084 26415 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:29:11.509281 26415 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:37937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.11.212:34867/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:38533
--ranger_config_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:38533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=45193
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26415
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:29:11.510270 26415 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:29:11.511261 26415 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:29:11.519582 26426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:11.519593 26423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:29:11.519675 26415 server_base.cc:1034] running on GCE node
W20260808 22:29:11.519593 26429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:29:11.520452 26415 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:29:11.521543 26415 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:29:11.522727 26415 hybrid_clock.cc:648] HybridClock initialized: now 1786228151522706 us; error 33 us; skew 500 ppm
Aug 08 22:29:11 dist-test-slave-7f01 krb5kdc[24846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228151, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:29:11.525959 26415 init.cc:377] Logged in from keytab as kudu/127.5.11.254@KRBTEST.COM (short username kudu)
I20260808 22:29:11.526839 26415 webserver.cc:460] Webserver started at http://127.5.11.254:45193/ using document root <none> and password file <none>
I20260808 22:29:11.527379 26415 fs_manager.cc:359] Metadata directory not provided
I20260808 22:29:11.527477 26415 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 22:29:11 dist-test-slave-7f01 krb5kdc[24846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228151, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for HTTP/127.5.11.212@KRBTEST.COM
I20260808 22:29:11.614945 26415 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 22:29:11.617708 26438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:11.619249 26415 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 22:29:11.619387 26415 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "a8ebf5ce638449f8a562ef695010caf5"
format_stamp: "Formatted at 2026-08-08 22:29:09 on dist-test-slave-7f01"
server_key: "2c109f24e30eb07c8752f900187360f8"
server_key_iv: "0d79df9b29d779cf78978e86684c2b38"
server_key_version: "kuduclusterkey@0"
I20260808 22:29:11.620018 26415 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:29:11.641458 26415 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:29:11.644079 26415 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:29:11.644273 26415 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:29:11.651413 26415 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:38533
I20260808 22:29:11.651423 26517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:38533 every 8 connection(s)
I20260808 22:29:11.652547 26415 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:29:11.653409 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 26415
I20260808 22:29:11.653959 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.11.212:34867/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.193:38029
--local_ip_for_outbound_sockets=127.5.11.193
--tserver_master_addrs=127.5.11.254:38533
--webserver_port=39447
--webserver_interface=127.5.11.193
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:37937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.11.212:34867/kms with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:29:11.657891 26518 sys_catalog.cc:263] Verifying existing consensus state
I20260808 22:29:11.659658 26518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5: Bootstrap starting.
I20260808 22:29:11.676266 26518 log.cc:824] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5: Log is configured to *not* fsync() on all Append() calls
I20260808 22:29:11.682821 26518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 22:29:11.683234 26518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5: Bootstrap complete.
I20260808 22:29:11.687486 26518 raft_consensus.cc:348] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ebf5ce638449f8a562ef695010caf5" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 38533 } }
I20260808 22:29:11.687927 26518 raft_consensus.cc:729] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a8ebf5ce638449f8a562ef695010caf5, State: Initialized, Role: FOLLOWER
I20260808 22:29:11.688447 26518 consensus_queue.cc:260] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ebf5ce638449f8a562ef695010caf5" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 38533 } }
I20260808 22:29:11.688588 26518 raft_consensus.cc:388] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:29:11.688668 26518 raft_consensus.cc:482] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:29:11.688769 26518 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5 [term 1 FOLLOWER]: Advancing to term 2
I20260808 22:29:11.690665 26518 raft_consensus.cc:504] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ebf5ce638449f8a562ef695010caf5" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 38533 } }
I20260808 22:29:11.691001 26518 leader_election.cc:302] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a8ebf5ce638449f8a562ef695010caf5; no voters:
I20260808 22:29:11.691354 26518 leader_election.cc:288] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 22:29:11.691473 26524 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 22:29:11.691727 26524 raft_consensus.cc:686] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5 [term 2 LEADER]: Becoming Leader. State: Replica: a8ebf5ce638449f8a562ef695010caf5, State: Running, Role: LEADER
I20260808 22:29:11.692133 26524 consensus_queue.cc:237] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ebf5ce638449f8a562ef695010caf5" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 38533 } }
I20260808 22:29:11.692641 26518 sys_catalog.cc:564] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:29:11.693693 26526 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a8ebf5ce638449f8a562ef695010caf5. Latest consensus state: current_term: 2 leader_uuid: "a8ebf5ce638449f8a562ef695010caf5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ebf5ce638449f8a562ef695010caf5" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 38533 } } }
I20260808 22:29:11.693697 26525 sys_catalog.cc:455] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a8ebf5ce638449f8a562ef695010caf5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a8ebf5ce638449f8a562ef695010caf5" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 38533 } } }
I20260808 22:29:11.693827 26526 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5 [sys.catalog]: This master's current role is: LEADER
I20260808 22:29:11.693840 26525 sys_catalog.cc:458] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5 [sys.catalog]: This master's current role is: LEADER
I20260808 22:29:11.694182 26538 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:29:11.697547 26538 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:29:11.698082 26518 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260808 22:29:11.698236 26538 catalog_manager.cc:1259] Loaded cluster ID: 3ca0e85d464c46668a7d7942fe761356
I20260808 22:29:11.698289 26538 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:29:11.703938 26538 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:29:11.705497 26538 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5: Loaded TSK: 0
I20260808 22:29:11.707167 26538 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 22:29:11.776109 26520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:29:11.776472 26520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:29:11.776539 26520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:29:11.782266 26520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:29:11.782354 26520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:29:11.782461 26520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.193
I20260808 22:29:11.788417 26520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:37937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.11.212:34867/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.193:38029
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.11.193
--webserver_port=39447
--tserver_master_addrs=127.5.11.254:38533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.193
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:29:11.789963 26520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:29:11.793074 26520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:29:11.808564 26585 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:11.808923 26587 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:29:11.809206 26520 server_base.cc:1034] running on GCE node
W20260808 22:29:11.809558 26584 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:29:11.810102 26520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:29:11.810814 26520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:29:11.819536 26520 hybrid_clock.cc:648] HybridClock initialized: now 1786228151819502 us; error 44 us; skew 500 ppm
Aug 08 22:29:11 dist-test-slave-7f01 krb5kdc[24846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228151, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:29:11.823776 26520 init.cc:377] Logged in from keytab as kudu/127.5.11.193@KRBTEST.COM (short username kudu)
I20260808 22:29:11.825073 26520 webserver.cc:460] Webserver started at http://127.5.11.193:39447/ using document root <none> and password file <none>
I20260808 22:29:11.825807 26520 fs_manager.cc:359] Metadata directory not provided
I20260808 22:29:11.825968 26520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 22:29:11 dist-test-slave-7f01 krb5kdc[24846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228151, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for HTTP/127.5.11.212@KRBTEST.COM
I20260808 22:29:11.859817 26520 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.003s sys 0.001s
I20260808 22:29:11.863112 26603 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:11.865041 26520 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260808 22:29:11.865176 26520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-0/wal
uuid: "a6817d6abcac41469590f9b7d73710f2"
format_stamp: "Formatted at 2026-08-08 22:29:10 on dist-test-slave-7f01"
server_key: "189d5395d596b78a54374a5f031aa1af"
server_key_iv: "ed2b139b1e9e0b7de1dca7290711637b"
server_key_version: "kuduclusterkey@0"
I20260808 22:29:11.865921 26520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:29:11.899889 26520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:29:11.903545 26520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:29:11.903818 26520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:29:11.904599 26520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:29:11.905637 26520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:29:11.905779 26520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:11.905925 26520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:29:11.906011 26520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:11.917268 26520 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.193:38029
I20260808 22:29:11.918751 26520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
I20260808 22:29:11.919044 26780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.193:38029 every 8 connection(s)
I20260808 22:29:11.924762 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 26520
I20260808 22:29:11.925261 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.11.212:34867/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.194:46345
--local_ip_for_outbound_sockets=127.5.11.194
--tserver_master_addrs=127.5.11.254:38533
--webserver_port=43009
--webserver_interface=127.5.11.194
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:37937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.11.212:34867/kms with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 22:29:11 dist-test-slave-7f01 krb5kdc[24846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228151, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:29:11.944051 26786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:11.921756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:38533 (local address 127.5.11.193:33973)
0808 22:29:11.922643 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:11.922705 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:11.924112 (+ 1407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:11.926547 (+ 2435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:11.926558 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:11.926991 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:29:11.927724 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:11.927789 (+ 65us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:11.929669 (+ 1880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:11.929676 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:11.931290 (+ 1614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:11.931305 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:11.932528 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:11.932532 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:11.932730 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:11.933406 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:29:11.933427 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:29:11.935237 (+ 1810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:29:11.938824 (+ 3587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:11.938830 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:11.938844 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:11.939415 (+ 571us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:11.941104 (+ 1689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:11.941107 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:11.941109 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:11.941294 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:11.942113 (+ 819us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:29:11.942119 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:29:11.942407 (+ 288us) client_negotiation.cc:769] Sending connection context
0808 22:29:11.942617 (+ 210us) client_negotiation.cc:240] Negotiation successful
0808 22:29:11.942959 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"thread_start_us":137,"threads_started":1}
I20260808 22:29:11.945713 26785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:11.921630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:33973 (local address 127.5.11.254:38533)
0808 22:29:11.922298 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:11.922324 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:11.923813 (+ 1489us) server_negotiation.cc:407] Connection header received
0808 22:29:11.924762 (+ 949us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:11.924778 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:11.925301 (+ 523us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:11.925962 (+ 661us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:11.927921 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:11.929509 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:11.931453 (+ 1944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:11.932328 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:11.932515 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:11.935473 (+ 2958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:11.935480 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:11.935496 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:11.935526 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:11.938685 (+ 3159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:11.939556 (+ 871us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:11.939562 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:11.939567 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:11.940974 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:11.941418 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:11.941424 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:11.941428 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:11.941922 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:11.942045 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:11.944579 (+ 2534us) server_negotiation.cc:299] Negotiation successful
0808 22:29:11.944900 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":136,"threads_started":1}
I20260808 22:29:11.949214 26781 heartbeater.cc:347] Connected to a master server at 127.5.11.254:38533
I20260808 22:29:11.949541 26781 heartbeater.cc:464] Registering TS with master...
I20260808 22:29:11.950178 26781 heartbeater.cc:511] Master 127.5.11.254:38533 requested a full tablet report, sending...
W20260808 22:29:11.952684 26781 heartbeater.cc:644] Failed to heartbeat to 127.5.11.254:38533 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 22:29:11.952826 26781 heartbeater.cc:464] Registering TS with master...
I20260808 22:29:11.953015 26781 heartbeater.cc:511] Master 127.5.11.254:38533 requested a full tablet report, sending...
I20260808 22:29:11.954063 26781 heartbeater.cc:464] Registering TS with master...
I20260808 22:29:11.954259 26781 heartbeater.cc:511] Master 127.5.11.254:38533 requested a full tablet report, sending...
W20260808 22:29:11.955194 26781 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 22:29:12.117235 26787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:29:12.117672 26787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:29:12.117792 26787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:29:12.127514 26787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:29:12.127660 26787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:29:12.127868 26787 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.194
I20260808 22:29:12.139599 26787 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:37937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.11.212:34867/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.194:46345
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.11.194
--webserver_port=43009
--tserver_master_addrs=127.5.11.254:38533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.194
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:29:12.141206 26787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:29:12.142812 26787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:29:12.157774 26800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:29:12.157823 26787 server_base.cc:1034] running on GCE node
W20260808 22:29:12.157634 26798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:12.158625 26795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:29:12.159214 26787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:29:12.160032 26787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:29:12.161226 26787 hybrid_clock.cc:648] HybridClock initialized: now 1786228152161205 us; error 40 us; skew 500 ppm
Aug 08 22:29:12 dist-test-slave-7f01 krb5kdc[24846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228152, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:29:12.164618 26787 init.cc:377] Logged in from keytab as kudu/127.5.11.194@KRBTEST.COM (short username kudu)
I20260808 22:29:12.165562 26787 webserver.cc:460] Webserver started at http://127.5.11.194:43009/ using document root <none> and password file <none>
I20260808 22:29:12.166136 26787 fs_manager.cc:359] Metadata directory not provided
I20260808 22:29:12.166240 26787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 22:29:12 dist-test-slave-7f01 krb5kdc[24846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228152, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for HTTP/127.5.11.212@KRBTEST.COM
I20260808 22:29:12.195290 26787 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 22:29:12.199013 26810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:12.200791 26787 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260808 22:29:12.200945 26787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-1/wal
uuid: "a0a9fc9ff93043b39554e5048156bfd3"
format_stamp: "Formatted at 2026-08-08 22:29:10 on dist-test-slave-7f01"
server_key: "fcf315238c750fa1e1d8e7acd5de7f22"
server_key_iv: "21cfdd372f70d4b2496526894228ee3d"
server_key_version: "kuduclusterkey@0"
I20260808 22:29:12.201766 26787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:29:12.231245 26787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:29:12.234412 26787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:29:12.234622 26787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:29:12.235376 26787 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:29:12.236510 26787 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:29:12.236636 26787 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:12.236753 26787 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:29:12.236826 26787 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:12.257055 26787 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.194:46345
I20260808 22:29:12.257515 26978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.194:46345 every 8 connection(s)
I20260808 22:29:12.258296 26787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
I20260808 22:29:12.265379 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 26787
I20260808 22:29:12.265897 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.5.11.212:34867/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.195:42989
--local_ip_for_outbound_sockets=127.5.11.195
--tserver_master_addrs=127.5.11.254:38533
--webserver_port=35219
--webserver_interface=127.5.11.195
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:37937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.5.11.212:34867/kms with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 22:29:12 dist-test-slave-7f01 krb5kdc[24846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228152, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:29:12.276137 26785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:12.260489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:40105 (local address 127.5.11.254:38533)
0808 22:29:12.260664 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:12.260669 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:12.262645 (+ 1976us) server_negotiation.cc:407] Connection header received
0808 22:29:12.263914 (+ 1269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:12.263919 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:12.263980 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:12.264079 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:12.265671 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:12.266520 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:12.268294 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:12.268662 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:12.268772 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:12.271658 (+ 2886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:12.271662 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:12.271665 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:12.271694 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:12.273644 (+ 1950us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:12.274276 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:12.274280 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:12.274282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:12.274375 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:12.274790 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:12.274793 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:12.274795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:12.275027 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:12.275101 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:12.275806 (+ 705us) server_negotiation.cc:299] Negotiation successful
0808 22:29:12.275963 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 22:29:12.277940 26981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:12.260833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:38533 (local address 127.5.11.194:40105)
0808 22:29:12.262448 (+ 1615us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:12.262514 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:12.263600 (+ 1086us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:12.264296 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:12.264306 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:12.264779 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:29:12.265485 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:12.265506 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:12.266687 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:12.266690 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:12.268156 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:12.268168 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:12.268965 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:12.268969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:12.269167 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:12.269832 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:29:12.269864 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:29:12.271454 (+ 1590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:29:12.273778 (+ 2324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:12.273785 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:12.273800 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:12.274153 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:12.274494 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:12.274497 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:12.274499 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:12.274674 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:12.275185 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:29:12.275191 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:29:12.275492 (+ 301us) client_negotiation.cc:769] Sending connection context
0808 22:29:12.276585 (+ 1093us) client_negotiation.cc:240] Negotiation successful
0808 22:29:12.276927 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":126,"threads_started":1}
I20260808 22:29:12.279436 26979 heartbeater.cc:347] Connected to a master server at 127.5.11.254:38533
I20260808 22:29:12.279698 26979 heartbeater.cc:464] Registering TS with master...
I20260808 22:29:12.280318 26979 heartbeater.cc:511] Master 127.5.11.254:38533 requested a full tablet report, sending...
W20260808 22:29:12.281983 26979 heartbeater.cc:644] Failed to heartbeat to 127.5.11.254:38533 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 22:29:12.282100 26979 heartbeater.cc:464] Registering TS with master...
I20260808 22:29:12.282248 26979 heartbeater.cc:511] Master 127.5.11.254:38533 requested a full tablet report, sending...
I20260808 22:29:12.283099 26979 heartbeater.cc:464] Registering TS with master...
I20260808 22:29:12.283258 26979 heartbeater.cc:511] Master 127.5.11.254:38533 requested a full tablet report, sending...
W20260808 22:29:12.284070 26979 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 22:29:12.438035 26983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:29:12.438462 26983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:29:12.438548 26983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:29:12.444643 26983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:29:12.444761 26983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:29:12.444875 26983 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.195
I20260808 22:29:12.452665 26983 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:37937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.5.11.212:34867/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.195:42989
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.11.195
--webserver_port=35219
--tserver_master_addrs=127.5.11.254:38533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.195
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:29:12.454208 26983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:29:12.455622 26983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:29:12.469177 26994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:12.469251 26992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:12.469334 26997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:29:12.469751 26983 server_base.cc:1034] running on GCE node
I20260808 22:29:12.470218 26983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:29:12.471017 26983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:29:12.474332 26983 hybrid_clock.cc:648] HybridClock initialized: now 1786228152474271 us; error 66 us; skew 500 ppm
Aug 08 22:29:12 dist-test-slave-7f01 krb5kdc[24846](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228152, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:29:12.478152 26983 init.cc:377] Logged in from keytab as kudu/127.5.11.195@KRBTEST.COM (short username kudu)
I20260808 22:29:12.479308 26983 webserver.cc:460] Webserver started at http://127.5.11.195:35219/ using document root <none> and password file <none>
I20260808 22:29:12.480089 26983 fs_manager.cc:359] Metadata directory not provided
I20260808 22:29:12.480206 26983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 08 22:29:12 dist-test-slave-7f01 krb5kdc[24846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228152, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for HTTP/127.5.11.212@KRBTEST.COM
I20260808 22:29:12.504429 26983 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 22:29:12.511672 27010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:12.513481 26983 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.001s sys 0.000s
I20260808 22:29:12.513705 26983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-2/wal
uuid: "0ebf78ad79c44a4f82b68aa49e9f1ac0"
format_stamp: "Formatted at 2026-08-08 22:29:11 on dist-test-slave-7f01"
server_key: "bc67cb4120b59d5ea0db97f616184dd4"
server_key_iv: "67eaf319254a56539afc0626b3ff7ec7"
server_key_version: "kuduclusterkey@0"
I20260808 22:29:12.514549 26983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:29:12.529835 26983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:29:12.533445 26983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:29:12.533697 26983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:29:12.534431 26983 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:29:12.535554 26983 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:29:12.535624 26983 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:12.535677 26983 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:29:12.535717 26983 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:12.548446 26983 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.195:42989
I20260808 22:29:12.550315 26983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
I20260808 22:29:12.550606 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 26983
I20260808 22:29:12.551810 27189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.195:42989 every 8 connection(s)
Aug 08 22:29:12 dist-test-slave-7f01 krb5kdc[24846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228152, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:29:12.573946 27198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:12.557946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:33893 (local address 127.5.11.254:38533)
0808 22:29:12.558232 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:12.558237 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:12.561042 (+ 2805us) server_negotiation.cc:407] Connection header received
0808 22:29:12.561105 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:12.561108 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:12.561164 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:12.561274 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:12.562530 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:12.563370 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:12.565156 (+ 1786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:12.565516 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:12.565614 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:12.568736 (+ 3122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:12.568740 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:12.568742 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:12.568766 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:12.570505 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:12.572291 (+ 1786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:12.572294 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:12.572295 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:12.572375 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:12.572831 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:12.572834 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:12.572836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:12.573013 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:12.573120 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:12.573614 (+ 494us) server_negotiation.cc:299] Negotiation successful
0808 22:29:12.573737 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":92,"threads_started":1}
I20260808 22:29:12.575325 27200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:12.558498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:38533 (local address 127.5.11.195:33893)
0808 22:29:12.559663 (+ 1165us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:12.559717 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:12.560787 (+ 1070us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:12.561495 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:12.561501 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:12.561774 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:29:12.562349 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:12.562371 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:12.563560 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:12.563564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:12.565013 (+ 1449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:12.565025 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:12.565807 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:12.565812 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:12.566005 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:12.566662 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:29:12.566678 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:29:12.568587 (+ 1909us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:29:12.571732 (+ 3145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:12.571759 (+ 27us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:12.571773 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:12.572170 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:12.572517 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:12.572520 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:12.572522 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:12.572712 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:12.573118 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:29:12.573126 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:29:12.573376 (+ 250us) client_negotiation.cc:769] Sending connection context
0808 22:29:12.573555 (+ 179us) client_negotiation.cc:240] Negotiation successful
0808 22:29:12.574175 (+ 620us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":955,"thread_start_us":147,"threads_started":1}
I20260808 22:29:12.576534 27192 heartbeater.cc:347] Connected to a master server at 127.5.11.254:38533
I20260808 22:29:12.576752 27192 heartbeater.cc:464] Registering TS with master...
I20260808 22:29:12.577186 27192 heartbeater.cc:511] Master 127.5.11.254:38533 requested a full tablet report, sending...
I20260808 22:29:12.578706 26785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:12.555971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59054 (local address 127.5.11.254:38533)
0808 22:29:12.556116 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:12.556123 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:12.568098 (+ 11975us) server_negotiation.cc:407] Connection header received
0808 22:29:12.568277 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:12.568281 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:12.568327 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:12.568402 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:12.569441 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:12.570505 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:12.571684 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:12.571957 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:12.572030 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:12.573890 (+ 1860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:12.573894 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:12.573896 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:12.573918 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:12.575515 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:12.576165 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:12.576170 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:12.576171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:12.576272 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:12.577669 (+ 1397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:12.577672 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:12.577674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:12.577890 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:12.578090 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:12.578420 (+ 330us) server_negotiation.cc:299] Negotiation successful
0808 22:29:12.578543 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260808 22:29:12.579126 27192 heartbeater.cc:644] Failed to heartbeat to 127.5.11.254:38533 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 22:29:12.579250 27192 heartbeater.cc:464] Registering TS with master...
I20260808 22:29:12.579388 27192 heartbeater.cc:511] Master 127.5.11.254:38533 requested a full tablet report, sending...
I20260808 22:29:12.580211 27192 heartbeater.cc:464] Registering TS with master...
I20260808 22:29:12.580351 27192 heartbeater.cc:511] Master 127.5.11.254:38533 requested a full tablet report, sending...
W20260808 22:29:12.581259 27192 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 08 22:29:12 dist-test-slave-7f01 krb5kdc[24846](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786228152, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:29:12.955478 26781 heartbeater.cc:464] Registering TS with master...
I20260808 22:29:12.955840 26781 heartbeater.cc:511] Master 127.5.11.254:38533 requested a full tablet report, sending...
I20260808 22:29:12.957170 26466 ts_manager.cc:194] Registered new tserver with Master: a6817d6abcac41469590f9b7d73710f2 (127.5.11.193:38029)
I20260808 22:29:12.958832 26466 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.193@KRBTEST.COM'} at 127.5.11.193:33973
I20260808 22:29:13.284298 26979 heartbeater.cc:464] Registering TS with master...
I20260808 22:29:13.284621 26979 heartbeater.cc:511] Master 127.5.11.254:38533 requested a full tablet report, sending...
I20260808 22:29:13.285392 26466 ts_manager.cc:194] Registered new tserver with Master: a0a9fc9ff93043b39554e5048156bfd3 (127.5.11.194:46345)
I20260808 22:29:13.286294 26466 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.194@KRBTEST.COM'} at 127.5.11.194:40105
Aug 08 22:29:13 dist-test-slave-7f01 krb5kdc[24846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228152, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for HTTP/127.5.11.212@KRBTEST.COM
Aug 08 22:29:13 dist-test-slave-7f01 krb5kdc[24846](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228152, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for HTTP/127.5.11.212@KRBTEST.COM
I20260808 22:29:13.581513 27192 heartbeater.cc:464] Registering TS with master...
I20260808 22:29:13.581840 27192 heartbeater.cc:511] Master 127.5.11.254:38533 requested a full tablet report, sending...
I20260808 22:29:13.582705 26466 ts_manager.cc:194] Registered new tserver with Master: 0ebf78ad79c44a4f82b68aa49e9f1ac0 (127.5.11.195:42989)
I20260808 22:29:13.583523 26466 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.195@KRBTEST.COM'} at 127.5.11.195:33893
I20260808 22:29:13.591495 5167 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:29:13.620756 27232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:13.593526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59056 (local address 127.5.11.254:38533)
0808 22:29:13.595504 (+ 1978us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:13.595509 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:13.596128 (+ 619us) server_negotiation.cc:407] Connection header received
0808 22:29:13.596957 (+ 829us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:13.596960 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:13.597017 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:13.597588 (+ 571us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:13.598936 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:13.601414 (+ 2478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:13.602831 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:13.603172 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:13.605154 (+ 1982us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:13.608349 (+ 3195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:13.608353 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:13.608356 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:13.608386 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:13.610425 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:13.610964 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:13.610967 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:13.610969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:13.611073 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:13.611830 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:13.611833 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:13.611835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:13.612076 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:13.612215 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:13.613346 (+ 1131us) server_negotiation.cc:299] Negotiation successful
0808 22:29:13.613490 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1892,"thread_start_us":88,"threads_started":1}
I20260808 22:29:13.625976 26466 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59056:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:29:13.631553 26466 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:29:13.700124 27247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:13.686375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:46345 (local address 127.0.0.1:49968)
0808 22:29:13.691438 (+ 5063us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:13.691452 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:13.691555 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:13.692204 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:13.692207 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:13.692227 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:29:13.692328 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:13.692335 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:13.694292 (+ 1957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:13.694296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:13.698106 (+ 3810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:13.698119 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:13.699491 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:13.699494 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:13.699626 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:13.699670 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 22:29:13.699810 (+ 140us) client_negotiation.cc:240] Negotiation successful
0808 22:29:13.699957 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4998,"mutex_wait_us":2206,"thread_start_us":116,"threads_started":1}
I20260808 22:29:13.700588 27249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:13.686844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49968 (local address 127.5.11.194:46345)
0808 22:29:13.691287 (+ 4443us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:13.691294 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:13.691661 (+ 367us) server_negotiation.cc:407] Connection header received
0808 22:29:13.691773 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:13.691779 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:13.691984 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:13.692721 (+ 737us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:29:13.692864 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:13.694147 (+ 1283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:13.698275 (+ 4128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:13.699159 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:13.699269 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:13.699423 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:13.700334 (+ 911us) server_negotiation.cc:299] Negotiation successful
0808 22:29:13.700444 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4333,"thread_start_us":111,"threads_started":1}
I20260808 22:29:13.702111 27250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:13.684453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:42989 (local address 127.0.0.1:34922)
0808 22:29:13.689497 (+ 5044us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:13.689550 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:13.689694 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:13.690739 (+ 1045us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:13.690746 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:13.690771 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:29:13.690921 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:13.690930 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:13.695483 (+ 4553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:13.695486 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:13.697472 (+ 1986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:13.697485 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:13.701706 (+ 4221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:13.701709 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:13.701795 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:13.701817 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 22:29:13.701912 (+ 95us) client_negotiation.cc:240] Negotiation successful
0808 22:29:13.701980 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4857,"thread_start_us":127,"threads_started":1}
I20260808 22:29:13.702658 27244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:13.684594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34922 (local address 127.5.11.195:42989)
0808 22:29:13.686922 (+ 2328us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:13.686931 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:13.690063 (+ 3132us) server_negotiation.cc:407] Connection header received
0808 22:29:13.690154 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:13.690160 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:13.690382 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:13.690542 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:29:13.691116 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:13.693203 (+ 2087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:13.700767 (+ 7564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:13.701555 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:13.701702 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:13.701838 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:13.702413 (+ 575us) server_negotiation.cc:299] Negotiation successful
0808 22:29:13.702515 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2189,"thread_start_us":135,"threads_started":1}
I20260808 22:29:13.704501 26882 tablet_service.cc:1467] Processing CreateTablet for tablet 7a88d76b76604e558e944a3372e989b2 (DEFAULT_TABLE table=test-table [id=9d23331655dd4747b9d011147bf74bd0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:29:13.709009 27088 tablet_service.cc:1467] Processing CreateTablet for tablet 7a88d76b76604e558e944a3372e989b2 (DEFAULT_TABLE table=test-table [id=9d23331655dd4747b9d011147bf74bd0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:29:13.710178 27088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a88d76b76604e558e944a3372e989b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:29:13.712779 26882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a88d76b76604e558e944a3372e989b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:29:13.723424 27243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:13.687608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:38029 (local address 127.0.0.1:34796)
0808 22:29:13.692533 (+ 4925us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:13.692545 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:13.692630 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:13.693955 (+ 1325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:13.693959 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:13.693980 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:29:13.694068 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:13.694074 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:13.697614 (+ 3540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:13.697618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:13.698734 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:13.698746 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:13.722896 (+ 24150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:13.722901 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:13.723034 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:13.723070 (+ 36us) client_negotiation.cc:769] Sending connection context
0808 22:29:13.723167 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 22:29:13.723263 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4865,"thread_start_us":101,"threads_started":1}
I20260808 22:29:13.723984 27251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:13.688096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34796 (local address 127.5.11.193:38029)
0808 22:29:13.690004 (+ 1908us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:13.690011 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:13.693369 (+ 3358us) server_negotiation.cc:407] Connection header received
0808 22:29:13.693452 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:13.693457 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:13.693641 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:13.693787 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:29:13.694209 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:13.695335 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:13.698914 (+ 3579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:13.699826 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:13.699951 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:13.700097 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:13.723659 (+ 23562us) server_negotiation.cc:299] Negotiation successful
0808 22:29:13.723820 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1785,"thread_start_us":70,"threads_started":1}
I20260808 22:29:13.724558 27252 tablet_bootstrap.cc:492] T 7a88d76b76604e558e944a3372e989b2 P 0ebf78ad79c44a4f82b68aa49e9f1ac0: Bootstrap starting.
I20260808 22:29:13.727164 27252 tablet_bootstrap.cc:654] T 7a88d76b76604e558e944a3372e989b2 P 0ebf78ad79c44a4f82b68aa49e9f1ac0: Neither blocks nor log segments found. Creating new log.
I20260808 22:29:13.727406 26673 tablet_service.cc:1467] Processing CreateTablet for tablet 7a88d76b76604e558e944a3372e989b2 (DEFAULT_TABLE table=test-table [id=9d23331655dd4747b9d011147bf74bd0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:29:13.728127 27252 log.cc:824] T 7a88d76b76604e558e944a3372e989b2 P 0ebf78ad79c44a4f82b68aa49e9f1ac0: Log is configured to *not* fsync() on all Append() calls
I20260808 22:29:13.728528 26673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a88d76b76604e558e944a3372e989b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:29:13.728842 27253 tablet_bootstrap.cc:492] T 7a88d76b76604e558e944a3372e989b2 P a0a9fc9ff93043b39554e5048156bfd3: Bootstrap starting.
I20260808 22:29:13.731027 27253 tablet_bootstrap.cc:654] T 7a88d76b76604e558e944a3372e989b2 P a0a9fc9ff93043b39554e5048156bfd3: Neither blocks nor log segments found. Creating new log.
I20260808 22:29:13.731856 27253 log.cc:824] T 7a88d76b76604e558e944a3372e989b2 P a0a9fc9ff93043b39554e5048156bfd3: Log is configured to *not* fsync() on all Append() calls
I20260808 22:29:13.734055 27252 tablet_bootstrap.cc:492] T 7a88d76b76604e558e944a3372e989b2 P 0ebf78ad79c44a4f82b68aa49e9f1ac0: No bootstrap required, opened a new log
I20260808 22:29:13.734274 27252 ts_tablet_manager.cc:1397] T 7a88d76b76604e558e944a3372e989b2 P 0ebf78ad79c44a4f82b68aa49e9f1ac0: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260808 22:29:13.738613 27252 raft_consensus.cc:348] T 7a88d76b76604e558e944a3372e989b2 P 0ebf78ad79c44a4f82b68aa49e9f1ac0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ebf78ad79c44a4f82b68aa49e9f1ac0" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 42989 } } peers { permanent_uuid: "a0a9fc9ff93043b39554e5048156bfd3" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 46345 } } peers { permanent_uuid: "a6817d6abcac41469590f9b7d73710f2" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 38029 } }
I20260808 22:29:13.738899 27252 raft_consensus.cc:374] T 7a88d76b76604e558e944a3372e989b2 P 0ebf78ad79c44a4f82b68aa49e9f1ac0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:29:13.738958 27252 raft_consensus.cc:729] T 7a88d76b76604e558e944a3372e989b2 P 0ebf78ad79c44a4f82b68aa49e9f1ac0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ebf78ad79c44a4f82b68aa49e9f1ac0, State: Initialized, Role: FOLLOWER
I20260808 22:29:13.739398 27252 consensus_queue.cc:260] T 7a88d76b76604e558e944a3372e989b2 P 0ebf78ad79c44a4f82b68aa49e9f1ac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ebf78ad79c44a4f82b68aa49e9f1ac0" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 42989 } } peers { permanent_uuid: "a0a9fc9ff93043b39554e5048156bfd3" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 46345 } } peers { permanent_uuid: "a6817d6abcac41469590f9b7d73710f2" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 38029 } }
I20260808 22:29:13.739848 27259 tablet_bootstrap.cc:492] T 7a88d76b76604e558e944a3372e989b2 P a6817d6abcac41469590f9b7d73710f2: Bootstrap starting.
I20260808 22:29:13.740492 27252 ts_tablet_manager.cc:1428] T 7a88d76b76604e558e944a3372e989b2 P 0ebf78ad79c44a4f82b68aa49e9f1ac0: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 22:29:13.741137 27192 heartbeater.cc:503] Master 127.5.11.254:38533 was elected leader, sending a full tablet report...
I20260808 22:29:13.742576 27253 tablet_bootstrap.cc:492] T 7a88d76b76604e558e944a3372e989b2 P a0a9fc9ff93043b39554e5048156bfd3: No bootstrap required, opened a new log
I20260808 22:29:13.742826 27253 ts_tablet_manager.cc:1397] T 7a88d76b76604e558e944a3372e989b2 P a0a9fc9ff93043b39554e5048156bfd3: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.000s
I20260808 22:29:13.746812 27259 tablet_bootstrap.cc:654] T 7a88d76b76604e558e944a3372e989b2 P a6817d6abcac41469590f9b7d73710f2: Neither blocks nor log segments found. Creating new log.
I20260808 22:29:13.747138 27253 raft_consensus.cc:348] T 7a88d76b76604e558e944a3372e989b2 P a0a9fc9ff93043b39554e5048156bfd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ebf78ad79c44a4f82b68aa49e9f1ac0" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 42989 } } peers { permanent_uuid: "a0a9fc9ff93043b39554e5048156bfd3" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 46345 } } peers { permanent_uuid: "a6817d6abcac41469590f9b7d73710f2" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 38029 } }
I20260808 22:29:13.747555 27253 raft_consensus.cc:374] T 7a88d76b76604e558e944a3372e989b2 P a0a9fc9ff93043b39554e5048156bfd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:29:13.747656 27253 raft_consensus.cc:729] T 7a88d76b76604e558e944a3372e989b2 P a0a9fc9ff93043b39554e5048156bfd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0a9fc9ff93043b39554e5048156bfd3, State: Initialized, Role: FOLLOWER
I20260808 22:29:13.747773 27259 log.cc:824] T 7a88d76b76604e558e944a3372e989b2 P a6817d6abcac41469590f9b7d73710f2: Log is configured to *not* fsync() on all Append() calls
I20260808 22:29:13.748183 27253 consensus_queue.cc:260] T 7a88d76b76604e558e944a3372e989b2 P a0a9fc9ff93043b39554e5048156bfd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ebf78ad79c44a4f82b68aa49e9f1ac0" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 42989 } } peers { permanent_uuid: "a0a9fc9ff93043b39554e5048156bfd3" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 46345 } } peers { permanent_uuid: "a6817d6abcac41469590f9b7d73710f2" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 38029 } }
I20260808 22:29:13.749187 27253 ts_tablet_manager.cc:1428] T 7a88d76b76604e558e944a3372e989b2 P a0a9fc9ff93043b39554e5048156bfd3: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 22:29:13.750154 27259 tablet_bootstrap.cc:492] T 7a88d76b76604e558e944a3372e989b2 P a6817d6abcac41469590f9b7d73710f2: No bootstrap required, opened a new log
I20260808 22:29:13.750334 27259 ts_tablet_manager.cc:1397] T 7a88d76b76604e558e944a3372e989b2 P a6817d6abcac41469590f9b7d73710f2: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.000s
I20260808 22:29:13.752400 26979 heartbeater.cc:503] Master 127.5.11.254:38533 was elected leader, sending a full tablet report...
I20260808 22:29:13.758029 27259 raft_consensus.cc:348] T 7a88d76b76604e558e944a3372e989b2 P a6817d6abcac41469590f9b7d73710f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ebf78ad79c44a4f82b68aa49e9f1ac0" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 42989 } } peers { permanent_uuid: "a0a9fc9ff93043b39554e5048156bfd3" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 46345 } } peers { permanent_uuid: "a6817d6abcac41469590f9b7d73710f2" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 38029 } }
I20260808 22:29:13.758375 27259 raft_consensus.cc:374] T 7a88d76b76604e558e944a3372e989b2 P a6817d6abcac41469590f9b7d73710f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:29:13.758486 27259 raft_consensus.cc:729] T 7a88d76b76604e558e944a3372e989b2 P a6817d6abcac41469590f9b7d73710f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6817d6abcac41469590f9b7d73710f2, State: Initialized, Role: FOLLOWER
I20260808 22:29:13.759059 27259 consensus_queue.cc:260] T 7a88d76b76604e558e944a3372e989b2 P a6817d6abcac41469590f9b7d73710f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ebf78ad79c44a4f82b68aa49e9f1ac0" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 42989 } } peers { permanent_uuid: "a0a9fc9ff93043b39554e5048156bfd3" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 46345 } } peers { permanent_uuid: "a6817d6abcac41469590f9b7d73710f2" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 38029 } }
I20260808 22:29:13.760229 27259 ts_tablet_manager.cc:1428] T 7a88d76b76604e558e944a3372e989b2 P a6817d6abcac41469590f9b7d73710f2: Time spent starting tablet: real 0.010s user 0.004s sys 0.003s
I20260808 22:29:13.760640 26781 heartbeater.cc:503] Master 127.5.11.254:38533 was elected leader, sending a full tablet report...
W20260808 22:29:13.761678 26980 tablet.cc:2369] T 7a88d76b76604e558e944a3372e989b2 P a0a9fc9ff93043b39554e5048156bfd3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 22:29:13.807360 27194 tablet.cc:2369] T 7a88d76b76604e558e944a3372e989b2 P 0ebf78ad79c44a4f82b68aa49e9f1ac0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 22:29:13.922516 26782 tablet.cc:2369] T 7a88d76b76604e558e944a3372e989b2 P a6817d6abcac41469590f9b7d73710f2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:29:14.053217 27262 raft_consensus.cc:482] T 7a88d76b76604e558e944a3372e989b2 P a0a9fc9ff93043b39554e5048156bfd3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:29:14.053544 27262 raft_consensus.cc:504] T 7a88d76b76604e558e944a3372e989b2 P a0a9fc9ff93043b39554e5048156bfd3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ebf78ad79c44a4f82b68aa49e9f1ac0" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 42989 } } peers { permanent_uuid: "a0a9fc9ff93043b39554e5048156bfd3" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 46345 } } peers { permanent_uuid: "a6817d6abcac41469590f9b7d73710f2" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 38029 } }
I20260808 22:29:14.054782 27262 leader_election.cc:288] T 7a88d76b76604e558e944a3372e989b2 P a0a9fc9ff93043b39554e5048156bfd3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ebf78ad79c44a4f82b68aa49e9f1ac0 (127.5.11.195:42989), a6817d6abcac41469590f9b7d73710f2 (127.5.11.193:38029)
I20260808 22:29:14.060348 27251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:14.055283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:59945 (local address 127.5.11.193:38029)
0808 22:29:14.055488 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:14.055492 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:14.055691 (+ 199us) server_negotiation.cc:407] Connection header received
0808 22:29:14.055845 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:14.055848 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:14.055896 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:14.055989 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:29:14.056361 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:14.057150 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:14.059062 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:14.059659 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:14.059823 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:14.059877 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:14.060145 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 22:29:14.060214 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260808 22:29:14.060348 27244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:14.055211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:44785 (local address 127.5.11.195:42989)
0808 22:29:14.055355 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:14.055360 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:14.055636 (+ 276us) server_negotiation.cc:407] Connection header received
0808 22:29:14.055819 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:14.055822 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:14.055880 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:14.055995 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:29:14.056361 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:14.057158 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:14.059060 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:14.059659 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:14.059823 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:14.059877 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:14.060145 (+ 268us) server_negotiation.cc:299] Negotiation successful
0808 22:29:14.060214 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 22:29:14.060412 27275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:14.055071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:38029 (local address 127.5.11.194:59945)
0808 22:29:14.055624 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:14.055636 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:14.055721 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:14.056022 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:14.056025 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:14.056046 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:29:14.056238 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:14.056246 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:14.057270 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:14.057274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:14.058927 (+ 1653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:14.058938 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:14.059795 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:14.059799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:14.059976 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:14.060034 (+ 58us) client_negotiation.cc:769] Sending connection context
0808 22:29:14.060115 (+ 81us) client_negotiation.cc:240] Negotiation successful
0808 22:29:14.060219 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"mutex_wait_us":313,"thread_start_us":135,"threads_started":1}
I20260808 22:29:14.060427 27273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:14.055071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:42989 (local address 127.5.11.194:44785)
0808 22:29:14.055522 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:14.055540 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:14.055675 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:14.056022 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:14.056025 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:14.056115 (+ 90us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:29:14.056242 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:14.056246 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:14.057270 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:14.057274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:14.058926 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:14.058938 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:14.059795 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:14.059799 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:14.059972 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:14.060035 (+ 63us) client_negotiation.cc:769] Sending connection context
0808 22:29:14.060115 (+ 80us) client_negotiation.cc:240] Negotiation successful
0808 22:29:14.060219 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":245,"spinlock_wait_cycles":38528,"thread_start_us":118,"threads_started":1}
I20260808 22:29:14.061316 27095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a88d76b76604e558e944a3372e989b2" candidate_uuid: "a0a9fc9ff93043b39554e5048156bfd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ebf78ad79c44a4f82b68aa49e9f1ac0" is_pre_election: true
I20260808 22:29:14.061307 26710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a88d76b76604e558e944a3372e989b2" candidate_uuid: "a0a9fc9ff93043b39554e5048156bfd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6817d6abcac41469590f9b7d73710f2" is_pre_election: true
I20260808 22:29:14.061633 26710 raft_consensus.cc:2454] T 7a88d76b76604e558e944a3372e989b2 P a6817d6abcac41469590f9b7d73710f2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0a9fc9ff93043b39554e5048156bfd3 in term 0.
I20260808 22:29:14.061633 27095 raft_consensus.cc:2454] T 7a88d76b76604e558e944a3372e989b2 P 0ebf78ad79c44a4f82b68aa49e9f1ac0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a0a9fc9ff93043b39554e5048156bfd3 in term 0.
I20260808 22:29:14.062105 26813 leader_election.cc:302] T 7a88d76b76604e558e944a3372e989b2 P a0a9fc9ff93043b39554e5048156bfd3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ebf78ad79c44a4f82b68aa49e9f1ac0, a0a9fc9ff93043b39554e5048156bfd3; no voters:
I20260808 22:29:14.062382 27262 raft_consensus.cc:2781] T 7a88d76b76604e558e944a3372e989b2 P a0a9fc9ff93043b39554e5048156bfd3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:29:14.062502 27262 raft_consensus.cc:482] T 7a88d76b76604e558e944a3372e989b2 P a0a9fc9ff93043b39554e5048156bfd3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:29:14.062561 27262 raft_consensus.cc:3037] T 7a88d76b76604e558e944a3372e989b2 P a0a9fc9ff93043b39554e5048156bfd3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:29:14.063597 27262 raft_consensus.cc:504] T 7a88d76b76604e558e944a3372e989b2 P a0a9fc9ff93043b39554e5048156bfd3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ebf78ad79c44a4f82b68aa49e9f1ac0" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 42989 } } peers { permanent_uuid: "a0a9fc9ff93043b39554e5048156bfd3" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 46345 } } peers { permanent_uuid: "a6817d6abcac41469590f9b7d73710f2" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 38029 } }
I20260808 22:29:14.063994 27262 leader_election.cc:288] T 7a88d76b76604e558e944a3372e989b2 P a0a9fc9ff93043b39554e5048156bfd3 [CANDIDATE]: Term 1 election: Requested vote from peers 0ebf78ad79c44a4f82b68aa49e9f1ac0 (127.5.11.195:42989), a6817d6abcac41469590f9b7d73710f2 (127.5.11.193:38029)
I20260808 22:29:14.064293 27095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a88d76b76604e558e944a3372e989b2" candidate_uuid: "a0a9fc9ff93043b39554e5048156bfd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ebf78ad79c44a4f82b68aa49e9f1ac0"
I20260808 22:29:14.064392 26710 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a88d76b76604e558e944a3372e989b2" candidate_uuid: "a0a9fc9ff93043b39554e5048156bfd3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6817d6abcac41469590f9b7d73710f2"
I20260808 22:29:14.064404 27095 raft_consensus.cc:3037] T 7a88d76b76604e558e944a3372e989b2 P 0ebf78ad79c44a4f82b68aa49e9f1ac0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:29:14.064543 26710 raft_consensus.cc:3037] T 7a88d76b76604e558e944a3372e989b2 P a6817d6abcac41469590f9b7d73710f2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:29:14.065539 27095 raft_consensus.cc:2454] T 7a88d76b76604e558e944a3372e989b2 P 0ebf78ad79c44a4f82b68aa49e9f1ac0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0a9fc9ff93043b39554e5048156bfd3 in term 1.
I20260808 22:29:14.065560 26710 raft_consensus.cc:2454] T 7a88d76b76604e558e944a3372e989b2 P a6817d6abcac41469590f9b7d73710f2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a0a9fc9ff93043b39554e5048156bfd3 in term 1.
I20260808 22:29:14.065872 26813 leader_election.cc:302] T 7a88d76b76604e558e944a3372e989b2 P a0a9fc9ff93043b39554e5048156bfd3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ebf78ad79c44a4f82b68aa49e9f1ac0, a0a9fc9ff93043b39554e5048156bfd3; no voters:
I20260808 22:29:14.066048 27262 raft_consensus.cc:2781] T 7a88d76b76604e558e944a3372e989b2 P a0a9fc9ff93043b39554e5048156bfd3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:29:14.066279 27262 raft_consensus.cc:686] T 7a88d76b76604e558e944a3372e989b2 P a0a9fc9ff93043b39554e5048156bfd3 [term 1 LEADER]: Becoming Leader. State: Replica: a0a9fc9ff93043b39554e5048156bfd3, State: Running, Role: LEADER
I20260808 22:29:14.066640 27262 consensus_queue.cc:237] T 7a88d76b76604e558e944a3372e989b2 P a0a9fc9ff93043b39554e5048156bfd3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ebf78ad79c44a4f82b68aa49e9f1ac0" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 42989 } } peers { permanent_uuid: "a0a9fc9ff93043b39554e5048156bfd3" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 46345 } } peers { permanent_uuid: "a6817d6abcac41469590f9b7d73710f2" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 38029 } }
I20260808 22:29:14.069895 26464 catalog_manager.cc:5495] T 7a88d76b76604e558e944a3372e989b2 P a0a9fc9ff93043b39554e5048156bfd3 reported cstate change: term changed from 0 to 1, leader changed from <none> to a0a9fc9ff93043b39554e5048156bfd3 (127.5.11.194). New cstate: current_term: 1 leader_uuid: "a0a9fc9ff93043b39554e5048156bfd3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ebf78ad79c44a4f82b68aa49e9f1ac0" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 42989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a0a9fc9ff93043b39554e5048156bfd3" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 46345 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6817d6abcac41469590f9b7d73710f2" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 38029 } health_report { overall_health: UNKNOWN } } }
I20260808 22:29:14.164683 27249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:14.161011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49978 (local address 127.5.11.194:46345)
0808 22:29:14.161140 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:14.161144 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:14.161350 (+ 206us) server_negotiation.cc:407] Connection header received
0808 22:29:14.161507 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:14.161509 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:14.161565 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:14.161662 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:29:14.161895 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:14.162362 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:14.163239 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:14.163452 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:14.163528 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:14.163857 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:29:14.164262 (+ 405us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:29:14.164353 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:14.164436 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 22:29:14.164485 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 22:29:14.170571 27095 raft_consensus.cc:1261] T 7a88d76b76604e558e944a3372e989b2 P 0ebf78ad79c44a4f82b68aa49e9f1ac0 [term 1 FOLLOWER]: Refusing update from remote peer a0a9fc9ff93043b39554e5048156bfd3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:29:14.170612 26710 raft_consensus.cc:1261] T 7a88d76b76604e558e944a3372e989b2 P a6817d6abcac41469590f9b7d73710f2 [term 1 FOLLOWER]: Refusing update from remote peer a0a9fc9ff93043b39554e5048156bfd3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:29:14.171373 27276 consensus_queue.cc:1035] T 7a88d76b76604e558e944a3372e989b2 P a0a9fc9ff93043b39554e5048156bfd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6817d6abcac41469590f9b7d73710f2" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 38029 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:29:14.171581 27262 consensus_queue.cc:1035] T 7a88d76b76604e558e944a3372e989b2 P a0a9fc9ff93043b39554e5048156bfd3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ebf78ad79c44a4f82b68aa49e9f1ac0" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 42989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:29:14.181113 27283 mvcc.cc:205] Tried to move back new op lower bound from 7316390519475392512 to 7316390519059599360. Current Snapshot: MvccSnapshot[applied={T|T < 7316390519475392512}]
I20260808 22:29:14.184583 27282 mvcc.cc:205] Tried to move back new op lower bound from 7316390519475392512 to 7316390519059599360. Current Snapshot: MvccSnapshot[applied={T|T < 7316390519475392512 or (T in {7316390519475392512})}]
I20260808 22:29:14.187500 26466 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:29:14.187668 26466 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59056:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:29:14.190124 26466 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a8ebf5ce638449f8a562ef695010caf5: Sending DeleteTablet for 3 replicas of tablet 7a88d76b76604e558e944a3372e989b2
I20260808 22:29:14.190811 27088 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a88d76b76604e558e944a3372e989b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:29:14 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:34922
I20260808 22:29:14.190827 26882 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a88d76b76604e558e944a3372e989b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:29:14 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:49968
I20260808 22:29:14.191346 27291 tablet_replica.cc:330] T 7a88d76b76604e558e944a3372e989b2 P a0a9fc9ff93043b39554e5048156bfd3: stopping tablet replica
I20260808 22:29:14.191717 27291 raft_consensus.cc:2229] T 7a88d76b76604e558e944a3372e989b2 P a0a9fc9ff93043b39554e5048156bfd3 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:29:14.191766 27292 tablet_replica.cc:330] T 7a88d76b76604e558e944a3372e989b2 P 0ebf78ad79c44a4f82b68aa49e9f1ac0: stopping tablet replica
I20260808 22:29:14.192034 27292 raft_consensus.cc:2229] T 7a88d76b76604e558e944a3372e989b2 P 0ebf78ad79c44a4f82b68aa49e9f1ac0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:29:14.192252 27291 raft_consensus.cc:2258] T 7a88d76b76604e558e944a3372e989b2 P a0a9fc9ff93043b39554e5048156bfd3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:29:14.192481 27292 raft_consensus.cc:2258] T 7a88d76b76604e558e944a3372e989b2 P 0ebf78ad79c44a4f82b68aa49e9f1ac0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:29:14.192847 26673 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a88d76b76604e558e944a3372e989b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:29:14 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:34796
I20260808 22:29:14.193504 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 26520
I20260808 22:29:14.194811 27292 ts_tablet_manager.cc:1905] T 7a88d76b76604e558e944a3372e989b2 P 0ebf78ad79c44a4f82b68aa49e9f1ac0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:29:14.198247 27291 ts_tablet_manager.cc:1905] T 7a88d76b76604e558e944a3372e989b2 P a0a9fc9ff93043b39554e5048156bfd3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:29:14.201064 27291 ts_tablet_manager.cc:1918] T 7a88d76b76604e558e944a3372e989b2 P a0a9fc9ff93043b39554e5048156bfd3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:29:14.201164 27291 log.cc:1196] T 7a88d76b76604e558e944a3372e989b2 P a0a9fc9ff93043b39554e5048156bfd3: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-1/wal/wals/7a88d76b76604e558e944a3372e989b2
I20260808 22:29:14.201500 27291 ts_tablet_manager.cc:1939] T 7a88d76b76604e558e944a3372e989b2 P a0a9fc9ff93043b39554e5048156bfd3: Deleting consensus metadata
I20260808 22:29:14.202620 26443 catalog_manager.cc:4841] TS a0a9fc9ff93043b39554e5048156bfd3 (127.5.11.194:46345): tablet 7a88d76b76604e558e944a3372e989b2 (table test-table [id=9d23331655dd4747b9d011147bf74bd0]) successfully deleted
I20260808 22:29:14.202914 27292 ts_tablet_manager.cc:1918] T 7a88d76b76604e558e944a3372e989b2 P 0ebf78ad79c44a4f82b68aa49e9f1ac0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:29:14.203009 27292 log.cc:1196] T 7a88d76b76604e558e944a3372e989b2 P 0ebf78ad79c44a4f82b68aa49e9f1ac0: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786227991988852-5167-0/minicluster-data/ts-2/wal/wals/7a88d76b76604e558e944a3372e989b2
I20260808 22:29:14.203313 27292 ts_tablet_manager.cc:1939] T 7a88d76b76604e558e944a3372e989b2 P 0ebf78ad79c44a4f82b68aa49e9f1ac0: Deleting consensus metadata
I20260808 22:29:14.204515 26442 catalog_manager.cc:4841] TS 0ebf78ad79c44a4f82b68aa49e9f1ac0 (127.5.11.195:42989): tablet 7a88d76b76604e558e944a3372e989b2 (table test-table [id=9d23331655dd4747b9d011147bf74bd0]) successfully deleted
W20260808 22:29:14.210255 26441 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.11.193:38029 (error 108)
I20260808 22:29:14.210433 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 26787
I20260808 22:29:14.212735 27243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:14.212382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:38029 (local address 127.0.0.1:34812)
0808 22:29:14.212550 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:14.212630 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.193:38029: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260808 22:29:14.212909 26441 catalog_manager.cc:4584] TS a6817d6abcac41469590f9b7d73710f2 (127.5.11.193:38029): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7a88d76b76604e558e944a3372e989b2: Network error: Client connection negotiation failed: client connection to 127.5.11.193:38029: connect: Connection refused (error 111)
I20260808 22:29:14.219713 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 26983
I20260808 22:29:14.229348 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 26415
I20260808 22:29:14.246212 5167 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260808 22:29:14.591499 5167 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260808 22:29:14.591642 5167 mini_ranger.cc:66] Stopping Ranger...
I20260808 22:29:14.936924 5167 mini_ranger.cc:68] Stopped Ranger
2026-08-08 22:29:14.937 UTC [24898] LOG: received smart shutdown request
2026-08-08 22:29:14.941 UTC [24898] LOG: background worker "logical replication launcher" (PID 24910) exited with exit code 1
2026-08-08 22:29:14.941 UTC [24903] LOG: shutting down
2026-08-08 22:29:15.100 UTC [24898] LOG: database system is shut down
2026-08-08T22:29:15Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49558 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:29:15 dist-test-slave-7f01 krb5kdc[27307](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:29:15 dist-test-slave-7f01 krb5kdc[27307](info): set up 2 sockets
Aug 08 22:29:15 dist-test-slave-7f01 krb5kdc[27307](info): commencing operation
krb5kdc: starting...
W20260808 22:29:16.484156 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.237s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:29:16 dist-test-slave-7f01 krb5kdc[27307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:29:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:29:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.11.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
I20260808 22:29:16.608414 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:42125
--webserver_interface=127.5.11.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:43967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:42125,127.5.11.253:42005,127.5.11.252:43267
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:29:16.718930 27322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:29:16.719157 27322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:29:16.719213 27322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:29:16.723006 27322 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:29:16.723054 27322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:29:16.723073 27322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:29:16.723089 27322 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:29:16.723104 27322 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:29:16.728047 27322 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:43967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:42125,127.5.11.253:42005,127.5.11.252:43267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:42125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:29:16.728915 27322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:29:16.729808 27322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:29:16.738013 27336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:16.738035 27331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:16.738051 27332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:29:16.738430 27322 server_base.cc:1034] running on GCE node
I20260808 22:29:16.739006 27322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:29:16.740067 27322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:29:16.741255 27322 hybrid_clock.cc:648] HybridClock initialized: now 1786228156741236 us; error 31 us; skew 500 ppm
Aug 08 22:29:16 dist-test-slave-7f01 krb5kdc[27307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228156, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:29:16.744088 27322 init.cc:377] Logged in from keytab as kudu/127.5.11.254@KRBTEST.COM (short username kudu)
I20260808 22:29:16.744894 27322 webserver.cc:460] Webserver started at http://127.5.11.254:44839/ using document root <none> and password file <none>
I20260808 22:29:16.745436 27322 fs_manager.cc:359] Metadata directory not provided
I20260808 22:29:16.745524 27322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:29:16.745746 27322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:29:16.747666 27322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "9f4328c9f9184ff4befe0245d945c3c1"
format_stamp: "Formatted at 2026-08-08 22:29:16 on dist-test-slave-7f01"
server_key: "311392ac554c624a090b9eb7f331ece5"
server_key_iv: "29733f59d668d459a0e6fc1116796426"
server_key_version: "encryptionkey@0"
I20260808 22:29:16.748279 27322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "9f4328c9f9184ff4befe0245d945c3c1"
format_stamp: "Formatted at 2026-08-08 22:29:16 on dist-test-slave-7f01"
server_key: "311392ac554c624a090b9eb7f331ece5"
server_key_iv: "29733f59d668d459a0e6fc1116796426"
server_key_version: "encryptionkey@0"
I20260808 22:29:16.751713 27322 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:29:16.754142 27347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:16.755297 27322 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:29:16.755438 27322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "9f4328c9f9184ff4befe0245d945c3c1"
format_stamp: "Formatted at 2026-08-08 22:29:16 on dist-test-slave-7f01"
server_key: "311392ac554c624a090b9eb7f331ece5"
server_key_iv: "29733f59d668d459a0e6fc1116796426"
server_key_version: "encryptionkey@0"
I20260808 22:29:16.755548 27322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:29:16.774611 27322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:29:16.777736 27322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:29:16.777973 27322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:29:16.784899 27322 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:42125
I20260808 22:29:16.784981 27424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:42125 every 8 connection(s)
I20260808 22:29:16.785817 27322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:29:16.788596 27426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:29:16.792320 27426 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.11.254" port: 42125 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:29:16.794323 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 27322
I20260808 22:29:16.794476 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:29:16.794792 5167 external_mini_cluster.cc:1428] Setting key 1b39b8867f6648602321b49dd91bc6cf
Aug 08 22:29:16 dist-test-slave-7f01 krb5kdc[27307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228156, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.11.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.253@KRBTEST.COM" created.
I20260808 22:29:16.807652 27430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:16.794492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:42125 (local address 127.0.0.1:47186)
0808 22:29:16.795058 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:16.795104 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:16.795978 (+ 874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:16.796730 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:16.796735 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:16.797228 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:29:16.797611 (+ 383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:16.797632 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:16.798919 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:16.798922 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:16.799622 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:16.799631 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:16.800584 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:16.800587 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:16.800679 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:16.801239 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:29:16.801255 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:29:16.802992 (+ 1737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:29:16.804978 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:16.804987 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:16.804995 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:16.805262 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:16.805554 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:16.805556 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:16.805557 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:16.805664 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:16.806198 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:29:16.806203 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:29:16.806329 (+ 126us) client_negotiation.cc:769] Sending connection context
0808 22:29:16.806476 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 22:29:16.806744 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"mutex_wait_us":1,"spinlock_wait_cycles":27904,"thread_start_us":160,"threads_started":1}
I20260808 22:29:16.807649 27429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:16.794492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47186 (local address 127.5.11.254:42125)
0808 22:29:16.794943 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:16.794963 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:16.795223 (+ 260us) server_negotiation.cc:407] Connection header received
0808 22:29:16.796276 (+ 1053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:16.796284 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:16.796519 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:16.796721 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:16.797774 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:16.798794 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:16.799774 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:16.800439 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:16.800612 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:16.803134 (+ 2522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:16.803139 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:16.803148 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:16.803168 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:16.804877 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:16.805356 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:16.805361 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:16.805362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:16.805474 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:16.805743 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:16.805745 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:16.805746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:16.806072 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:16.806174 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:16.806556 (+ 382us) server_negotiation.cc:299] Negotiation successful
0808 22:29:16.806849 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":118,"server-negotiator.queue_time_us":292,"thread_start_us":128,"threads_started":1}
I20260808 22:29:16.810317 27426 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.11.253" port: 42005 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:29:16.811017 27430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:16.810763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.253:42005 (local address 127.0.0.1:42310)
0808 22:29:16.810895 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:16.810967 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.253:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 22:29:16.811262 27350 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.11.253:42005: connect: Connection refused (error 111)
I20260808 22:29:16.812211 27430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:16.811999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.253:42005 (local address 127.0.0.1:42312)
0808 22:29:16.812118 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:16.812163 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.253:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 22:29:16.812407 27426 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.11.253:42005: Network error: Client connection negotiation failed: client connection to 127.5.11.253:42005: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.253.keytab.
Entry for principal kudu/127.5.11.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.253.keytab.
I20260808 22:29:16.823148 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.253:42005
--webserver_interface=127.5.11.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.253.keytab
--principal=kudu/127.5.11.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:43967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:42125,127.5.11.253:42005,127.5.11.252:43267
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:29:16.861673 27426 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.11.253" port: 42005 } attempt: 1
I20260808 22:29:16.862607 27430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:16.862293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.253:42005 (local address 127.0.0.1:42314)
0808 22:29:16.862459 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:16.862524 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.253:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260808 22:29:16.863330 27430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:16.863144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.253:42005 (local address 127.0.0.1:42318)
0808 22:29:16.863238 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:16.863281 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.253:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 22:29:16.863492 27426 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.11.253:42005: Network error: Client connection negotiation failed: client connection to 127.5.11.253:42005: connect: Connection refused (error 111)
I20260808 22:29:16.931684 27426 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.11.253" port: 42005 } attempt: 2
I20260808 22:29:16.932673 27430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:16.932338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.253:42005 (local address 127.0.0.1:42322)
0808 22:29:16.932522 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:16.932590 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.253:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260808 22:29:16.933491 27430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:16.933288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.253:42005 (local address 127.0.0.1:42332)
0808 22:29:16.933390 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:16.933437 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.253:42005: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 22:29:16.933638 27426 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.11.253:42005: Network error: Client connection negotiation failed: client connection to 127.5.11.253:42005: connect: Connection refused (error 111)
W20260808 22:29:16.934827 27434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:29:16.935041 27434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:29:16.935127 27434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:29:16.939092 27434 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:29:16.939141 27434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:29:16.939160 27434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:29:16.939177 27434 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:29:16.939229 27434 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:29:16.944253 27434 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:43967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:42125,127.5.11.253:42005,127.5.11.252:43267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.253.keytab
--principal=kudu/127.5.11.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.253:42005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.11.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:29:16.945169 27434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:29:16.946108 27434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:29:16.954360 27444 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:16.954367 27447 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:16.954348 27445 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:29:16.954687 27434 server_base.cc:1034] running on GCE node
I20260808 22:29:16.955281 27434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:29:16.956378 27434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:29:16.957567 27434 hybrid_clock.cc:648] HybridClock initialized: now 1786228156957552 us; error 27 us; skew 500 ppm
Aug 08 22:29:16 dist-test-slave-7f01 krb5kdc[27307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228156, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:29:16.960579 27434 init.cc:377] Logged in from keytab as kudu/127.5.11.253@KRBTEST.COM (short username kudu)
I20260808 22:29:16.961529 27434 webserver.cc:460] Webserver started at http://127.5.11.253:46713/ using document root <none> and password file <none>
I20260808 22:29:16.962095 27434 fs_manager.cc:359] Metadata directory not provided
I20260808 22:29:16.962188 27434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:29:16.962414 27434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:29:16.964490 27434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-1/data/instance:
uuid: "c69e70a4419e474aacdf3feadf9caf76"
format_stamp: "Formatted at 2026-08-08 22:29:16 on dist-test-slave-7f01"
server_key: "47e419f476e7da6b314ac62421bc41e9"
server_key_iv: "e2b500aa9f6357d99b60a967bc5b8a77"
server_key_version: "encryptionkey@0"
I20260808 22:29:16.965049 27434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-1/wal/instance:
uuid: "c69e70a4419e474aacdf3feadf9caf76"
format_stamp: "Formatted at 2026-08-08 22:29:16 on dist-test-slave-7f01"
server_key: "47e419f476e7da6b314ac62421bc41e9"
server_key_iv: "e2b500aa9f6357d99b60a967bc5b8a77"
server_key_version: "encryptionkey@0"
I20260808 22:29:16.968641 27434 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 22:29:16.970911 27457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:16.972128 27434 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:29:16.972273 27434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-1/wal
uuid: "c69e70a4419e474aacdf3feadf9caf76"
format_stamp: "Formatted at 2026-08-08 22:29:16 on dist-test-slave-7f01"
server_key: "47e419f476e7da6b314ac62421bc41e9"
server_key_iv: "e2b500aa9f6357d99b60a967bc5b8a77"
server_key_version: "encryptionkey@0"
I20260808 22:29:16.972381 27434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:29:16.985532 27434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:29:16.988204 27434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:29:16.988394 27434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:29:16.995496 27434 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.253:42005
I20260808 22:29:16.995509 27533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.253:42005 every 8 connection(s)
I20260808 22:29:16.996436 27434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-1/data/info.pb
I20260808 22:29:16.998945 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 27434
I20260808 22:29:16.999086 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-1/wal/instance
I20260808 22:29:16.999159 27534 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:29:16.999416 5167 external_mini_cluster.cc:1428] Setting key 6dce33de5ccdf0411b60ec0e0b966bc3
I20260808 22:29:17.003412 27534 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.11.254" port: 42125 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.5.11.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.252@KRBTEST.COM" created.
Aug 08 22:29:17 dist-test-slave-7f01 krb5kdc[27307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228156, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.253@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:29:17.018100 27429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:17.005345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47202 (local address 127.5.11.254:42125)
0808 22:29:17.005477 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:17.005480 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:17.006460 (+ 980us) server_negotiation.cc:407] Connection header received
0808 22:29:17.007440 (+ 980us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:17.007443 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:17.007513 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:17.007616 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:17.008771 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.009297 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.010545 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.010846 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.010952 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:17.013670 (+ 2718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:17.013678 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:17.013684 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:17.013718 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:17.015808 (+ 2090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:17.016370 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:17.016376 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:17.016381 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:17.016499 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:17.016840 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:17.016847 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:17.016851 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:17.017107 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:17.017223 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:17.017710 (+ 487us) server_negotiation.cc:299] Negotiation successful
0808 22:29:17.017865 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260808 22:29:17.018603 27538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:17.005826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:42125 (local address 127.0.0.1:47202)
0808 22:29:17.006311 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:17.006355 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:17.007228 (+ 873us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:17.007829 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:17.007842 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:17.008152 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:29:17.008592 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:17.008604 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.009454 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.009457 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:17.010411 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:17.010422 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.011003 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.011008 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:17.011167 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:17.011845 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:29:17.011862 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:29:17.013499 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:29:17.015936 (+ 2437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:17.015942 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:17.015955 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:17.016242 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:17.016606 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:17.016608 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:17.016610 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:17.016739 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:17.017225 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:29:17.017230 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:29:17.017451 (+ 221us) client_negotiation.cc:769] Sending connection context
0808 22:29:17.017598 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 22:29:17.017814 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":130,"threads_started":1}
I20260808 22:29:17.020018 27534 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.11.253" port: 42005 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:29:17.024824 27426 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.11.253" port: 42005 } attempt: 3
Aug 08 22:29:17 dist-test-slave-7f01 krb5kdc[27307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228156, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.253@KRBTEST.COM for kudu/127.5.11.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.252.keytab.
Entry for principal kudu/127.5.11.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.252.keytab.
I20260808 22:29:17.028966 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.252:43267
--webserver_interface=127.5.11.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.252.keytab
--principal=kudu/127.5.11.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:43967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:42125,127.5.11.253:42005,127.5.11.252:43267
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 22:29:17 dist-test-slave-7f01 krb5kdc[27307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228156, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for kudu/127.5.11.253@KRBTEST.COM
I20260808 22:29:17.030584 27538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:17.020506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.253:42005 (local address 127.0.0.1:42336)
0808 22:29:17.020608 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:17.020620 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:17.020678 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:17.021272 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:17.021275 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:17.021423 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:29:17.021504 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:17.021509 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.022485 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.022488 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:17.023149 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:17.023159 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.023868 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.023871 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:17.023972 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:17.024357 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:29:17.024391 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:29:17.025785 (+ 1394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:29:17.027968 (+ 2183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:17.027975 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:17.027980 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:17.028256 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:17.028963 (+ 707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:17.028967 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:17.028969 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:17.029072 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:17.030133 (+ 1061us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:29:17.030136 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:29:17.030216 (+ 80us) client_negotiation.cc:769] Sending connection context
0808 22:29:17.030302 (+ 86us) client_negotiation.cc:240] Negotiation successful
0808 22:29:17.030426 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260808 22:29:17.031774 27541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:17.020660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42336 (local address 127.5.11.253:42005)
0808 22:29:17.020966 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:17.020974 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:17.020991 (+ 17us) server_negotiation.cc:407] Connection header received
0808 22:29:17.021021 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:17.021024 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:17.021152 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:17.021249 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:17.021602 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.022361 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.023319 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.023716 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.023855 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:17.025929 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:17.025940 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:17.025952 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:17.025980 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:17.027855 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:17.028380 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:17.028385 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:17.028387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:17.028481 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:17.029702 (+ 1221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:17.029709 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:17.029714 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:17.030016 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:17.030846 (+ 830us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:17.030935 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 22:29:17.031036 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":154,"threads_started":1}
I20260808 22:29:17.033725 27534 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.11.252" port: 43267 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:29:17.034547 27538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:17.034189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:43267 (local address 127.0.0.1:57582)
0808 22:29:17.034387 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:17.034470 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.252:43267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
W20260808 22:29:17.034801 27461 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.11.252:43267: connect: Connection refused (error 111)
I20260808 22:29:17.034895 27430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:17.025209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.253:42005 (local address 127.0.0.1:42340)
0808 22:29:17.025339 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:17.025354 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:17.025429 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:17.025860 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:17.025864 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:17.026041 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:29:17.026162 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:17.026174 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.027213 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.027216 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:17.028067 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:17.028077 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.028603 (+ 526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.028609 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:17.028744 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:17.029211 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:29:17.029225 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:29:17.030550 (+ 1325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:29:17.033378 (+ 2828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:17.033381 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:17.033383 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:17.033576 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:17.033933 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:17.033937 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:17.033939 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:17.034032 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:17.034387 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:29:17.034390 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:29:17.034491 (+ 101us) client_negotiation.cc:769] Sending connection context
0808 22:29:17.034598 (+ 107us) client_negotiation.cc:240] Negotiation successful
0808 22:29:17.034724 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 22:29:17.034957 27543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:17.025300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42340 (local address 127.5.11.253:42005)
0808 22:29:17.025604 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:17.025612 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:17.025625 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:29:17.025659 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:17.025661 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:17.025707 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:17.025794 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:17.026303 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.027103 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.028191 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.028481 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.028584 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:17.030907 (+ 2323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:17.030914 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:17.030918 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:17.030959 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:17.033243 (+ 2284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:17.033703 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:17.033706 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:17.033707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:17.033809 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:17.034138 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:17.034140 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:17.034142 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:17.034286 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:17.034390 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:17.034651 (+ 261us) server_negotiation.cc:299] Negotiation successful
0808 22:29:17.034754 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":117,"threads_started":1}
I20260808 22:29:17.035660 27538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:17.035475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:43267 (local address 127.0.0.1:57598)
0808 22:29:17.035575 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:17.035616 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.252:43267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 22:29:17.035892 27534 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.11.252:43267: Network error: Client connection negotiation failed: client connection to 127.5.11.252:43267: connect: Connection refused (error 111)
I20260808 22:29:17.036089 27426 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.11.252" port: 43267 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:29:17.036700 27430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:17.036490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:43267 (local address 127.0.0.1:57608)
0808 22:29:17.036596 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:17.036651 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.252:43267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 22:29:17.037325 27430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:17.037103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:43267 (local address 127.0.0.1:57612)
0808 22:29:17.037231 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:17.037274 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.252:43267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
W20260808 22:29:17.037446 27426 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.11.252:43267: Network error: Client connection negotiation failed: client connection to 127.5.11.252:43267: connect: Connection refused (error 111)
I20260808 22:29:17.068607 27426 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.11.252" port: 43267 } attempt: 1
I20260808 22:29:17.069208 27430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:17.068984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:43267 (local address 127.0.0.1:57628)
0808 22:29:17.069100 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:17.069154 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.252:43267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260808 22:29:17.069883 27430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:17.069685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:43267 (local address 127.0.0.1:57634)
0808 22:29:17.069786 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:17.069838 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.252:43267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260808 22:29:17.070045 27426 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.11.252:43267: Network error: Client connection negotiation failed: client connection to 127.5.11.252:43267: connect: Connection refused (error 111)
I20260808 22:29:17.085110 27534 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.11.252" port: 43267 } attempt: 1
I20260808 22:29:17.085721 27538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:17.085491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:43267 (local address 127.0.0.1:57638)
0808 22:29:17.085611 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:17.085665 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.252:43267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 22:29:17.086390 27538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:17.086192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:43267 (local address 127.0.0.1:57650)
0808 22:29:17.086293 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:17.086336 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.252:43267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 22:29:17.086555 27534 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.11.252:43267: Network error: Client connection negotiation failed: client connection to 127.5.11.252:43267: connect: Connection refused (error 111)
W20260808 22:29:17.141811 27544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:29:17.142030 27544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:29:17.142066 27544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 22:29:17.145205 27426 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.11.252" port: 43267 } attempt: 2
I20260808 22:29:17.146039 27430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:17.145751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:43267 (local address 127.0.0.1:57654)
0808 22:29:17.145900 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:17.145965 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.252:43267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 22:29:17.146423 27544 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:29:17.146502 27544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:29:17.146545 27544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:29:17.146580 27544 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:29:17.146603 27544 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:29:17.146898 27430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:17.146632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:43267 (local address 127.0.0.1:57658)
0808 22:29:17.146790 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:17.146837 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.252:43267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260808 22:29:17.147035 27426 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.11.252:43267: Network error: Client connection negotiation failed: client connection to 127.5.11.252:43267: connect: Connection refused (error 111)
I20260808 22:29:17.152063 27544 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:43967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:42125,127.5.11.253:42005,127.5.11.252:43267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.252.keytab
--principal=kudu/127.5.11.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.252:43267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.11.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:29:17.152920 27544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:29:17.153796 27544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 22:29:17.154701 27534 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.11.252" port: 43267 } attempt: 2
I20260808 22:29:17.155510 27538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:17.155150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:43267 (local address 127.0.0.1:57662)
0808 22:29:17.155348 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:17.155403 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.252:43267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260808 22:29:17.156358 27538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:17.156114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:43267 (local address 127.0.0.1:57664)
0808 22:29:17.156227 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:17.156291 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.252:43267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260808 22:29:17.156500 27534 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.11.252:43267: Network error: Client connection negotiation failed: client connection to 127.5.11.252:43267: connect: Connection refused (error 111)
W20260808 22:29:17.162456 27554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:17.162552 27559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:17.162484 27553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:29:17.162868 27544 server_base.cc:1034] running on GCE node
I20260808 22:29:17.163347 27544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:29:17.164420 27544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:29:17.165583 27544 hybrid_clock.cc:648] HybridClock initialized: now 1786228157165571 us; error 22 us; skew 500 ppm
Aug 08 22:29:17 dist-test-slave-7f01 krb5kdc[27307](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228157, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:29:17.168272 27544 init.cc:377] Logged in from keytab as kudu/127.5.11.252@KRBTEST.COM (short username kudu)
I20260808 22:29:17.169073 27544 webserver.cc:460] Webserver started at http://127.5.11.252:33607/ using document root <none> and password file <none>
I20260808 22:29:17.169607 27544 fs_manager.cc:359] Metadata directory not provided
I20260808 22:29:17.169698 27544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:29:17.169935 27544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:29:17.171903 27544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-2/data/instance:
uuid: "ffe5072439ee40cb94b5cf7089ce3621"
format_stamp: "Formatted at 2026-08-08 22:29:17 on dist-test-slave-7f01"
server_key: "fac76c6a3949f28850e28d76f2e8fd75"
server_key_iv: "c77730161a33b44e384cb5371491e6b0"
server_key_version: "encryptionkey@0"
I20260808 22:29:17.172456 27544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-2/wal/instance:
uuid: "ffe5072439ee40cb94b5cf7089ce3621"
format_stamp: "Formatted at 2026-08-08 22:29:17 on dist-test-slave-7f01"
server_key: "fac76c6a3949f28850e28d76f2e8fd75"
server_key_iv: "c77730161a33b44e384cb5371491e6b0"
server_key_version: "encryptionkey@0"
I20260808 22:29:17.176127 27544 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
I20260808 22:29:17.178449 27570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:17.179569 27544 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:29:17.179706 27544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-2/wal
uuid: "ffe5072439ee40cb94b5cf7089ce3621"
format_stamp: "Formatted at 2026-08-08 22:29:17 on dist-test-slave-7f01"
server_key: "fac76c6a3949f28850e28d76f2e8fd75"
server_key_iv: "c77730161a33b44e384cb5371491e6b0"
server_key_version: "encryptionkey@0"
I20260808 22:29:17.179875 27544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:29:17.197444 27544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:29:17.199962 27544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:29:17.200146 27544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:29:17.206949 27544 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.252:43267
I20260808 22:29:17.206959 27638 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.252:43267 every 8 connection(s)
I20260808 22:29:17.207830 27544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-2/data/info.pb
I20260808 22:29:17.210462 27641 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:29:17.214015 27641 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.11.254" port: 42125 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:29:17.215757 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 27544
I20260808 22:29:17.215902 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786227991988852-5167-0/minicluster-data/master-2/wal/instance
I20260808 22:29:17.216183 5167 external_mini_cluster.cc:1428] Setting key d0ed46401363d8a27ac8a75cd8c2d75f
Aug 08 22:29:17 dist-test-slave-7f01 krb5kdc[27307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228157, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.252@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
Aug 08 22:29:17 dist-test-slave-7f01 krb5kdc[27307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.252@KRBTEST.COM
I20260808 22:29:17.228705 27429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:17.215900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48434 (local address 127.5.11.254:42125)
0808 22:29:17.216055 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:17.216059 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:17.216921 (+ 862us) server_negotiation.cc:407] Connection header received
0808 22:29:17.218000 (+ 1079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:17.218004 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:17.218053 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:17.218157 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:17.219216 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.220018 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.221458 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.221790 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.221900 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:17.224761 (+ 2861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:17.224764 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:17.224767 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:17.224791 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:17.226429 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:17.227097 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:17.227099 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:17.227100 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:17.227180 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:17.227563 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:17.227567 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:17.227569 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:17.227836 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:17.227947 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:17.228411 (+ 464us) server_negotiation.cc:299] Negotiation successful
0808 22:29:17.228530 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260808 22:29:17.229519 27646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:17.216871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38848 (local address 127.5.11.252:43267)
0808 22:29:17.217193 (+ 322us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:17.217202 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:17.217347 (+ 145us) server_negotiation.cc:407] Connection header received
0808 22:29:17.217827 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:17.217846 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:17.218064 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:17.218236 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:17.219106 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.220311 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.221458 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.222028 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.222108 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:17.225153 (+ 3045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:17.225158 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:17.225167 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:17.225188 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:17.227002 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:17.227452 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:17.227458 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:17.227464 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:17.227539 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:17.227843 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:17.227846 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:17.227848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:17.228171 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:17.228388 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:17.228554 (+ 166us) server_negotiation.cc:299] Negotiation successful
0808 22:29:17.228745 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":240,"spinlock_wait_cycles":2816,"thread_start_us":118,"threads_started":1}
I20260808 22:29:17.229523 27644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:17.216211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:42125 (local address 127.0.0.1:48434)
0808 22:29:17.216765 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:17.216820 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:17.217804 (+ 984us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:17.218239 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:17.218244 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:17.218709 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:29:17.219065 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:17.219085 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.220153 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.220156 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:17.221318 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:17.221330 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.221936 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.221939 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:17.222068 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:17.222689 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:29:17.222716 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:29:17.224587 (+ 1871us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:29:17.226541 (+ 1954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:17.226552 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:17.226561 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:17.226997 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:17.227284 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:17.227288 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:17.227290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:17.227438 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:17.227947 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:29:17.227954 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:29:17.228188 (+ 234us) client_negotiation.cc:769] Sending connection context
0808 22:29:17.228361 (+ 173us) client_negotiation.cc:240] Negotiation successful
0808 22:29:17.228679 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"spinlock_wait_cycles":1792,"thread_start_us":136,"threads_started":1}
I20260808 22:29:17.230813 27641 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.11.253" port: 42005 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 22:29:17 dist-test-slave-7f01 krb5kdc[27307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228157, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.252@KRBTEST.COM for kudu/127.5.11.253@KRBTEST.COM
Aug 08 22:29:17 dist-test-slave-7f01 krb5kdc[27307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:29:17.240191 27644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:17.231305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.253:42005 (local address 127.0.0.1:38638)
0808 22:29:17.231531 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:17.231542 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:17.231637 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:17.232091 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:17.232094 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:17.232271 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:29:17.232404 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:17.232415 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.233441 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.233443 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:17.234283 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:17.234294 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.234911 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.234915 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:17.235014 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:17.235540 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:29:17.235558 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:29:17.236861 (+ 1303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:29:17.238767 (+ 1906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:17.238771 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:17.238773 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:17.238966 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:17.239299 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:17.239302 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:17.239303 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:17.239385 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:17.239822 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:29:17.239825 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:29:17.239903 (+ 78us) client_negotiation.cc:769] Sending connection context
0808 22:29:17.239987 (+ 84us) client_negotiation.cc:240] Negotiation successful
0808 22:29:17.240087 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171}
I20260808 22:29:17.240358 27543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:17.231420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38638 (local address 127.5.11.253:42005)
0808 22:29:17.231575 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:17.231580 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:17.231600 (+ 20us) server_negotiation.cc:407] Connection header received
0808 22:29:17.231786 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:17.231800 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:17.231859 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:17.231915 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:17.232548 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.233330 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.234405 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.234768 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.234883 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:17.237002 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:17.237012 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:17.237015 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:17.237047 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:17.238654 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:17.239071 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:17.239074 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:17.239076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:17.239189 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:17.239481 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:17.239485 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:17.239487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:17.239711 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:17.239837 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:17.240067 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 22:29:17.240193 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260808 22:29:17.241147 27429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:17.231693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48440 (local address 127.5.11.254:42125)
0808 22:29:17.231818 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:17.231822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:17.231842 (+ 20us) server_negotiation.cc:407] Connection header received
0808 22:29:17.231940 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:17.231942 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:17.231975 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:17.232044 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:17.232765 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.233450 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.234508 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.234811 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.234903 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:17.237554 (+ 2651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:17.237558 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:17.237561 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:17.237591 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:17.239083 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:17.239516 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:17.239519 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:17.239520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:17.239592 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:17.239883 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:17.239892 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:17.239893 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:17.240099 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:17.240314 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:17.240886 (+ 572us) server_negotiation.cc:299] Negotiation successful
0808 22:29:17.240999 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260808 22:29:17.241140 27641 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.11.252" port: 43267 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:29:17.246181 27426 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.11.252" port: 43267 } attempt: 3
Aug 08 22:29:17 dist-test-slave-7f01 krb5kdc[27307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228157, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.252@KRBTEST.COM for kudu/127.5.11.252@KRBTEST.COM
Aug 08 22:29:17 dist-test-slave-7f01 krb5kdc[27307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228156, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.253@KRBTEST.COM
I20260808 22:29:17.247653 27534 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.11.252" port: 43267 } attempt: 3
I20260808 22:29:17.250751 27646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:17.241601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38858 (local address 127.5.11.252:43267)
0808 22:29:17.241690 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:17.241694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:17.241731 (+ 37us) server_negotiation.cc:407] Connection header received
0808 22:29:17.241852 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:17.241855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:17.241892 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:17.241972 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:17.242380 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.243192 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.244257 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.244612 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.244713 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:17.246831 (+ 2118us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:17.246836 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:17.246842 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:17.246871 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:17.248611 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:17.249546 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:17.249552 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:17.249557 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:17.249662 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:17.249958 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:17.249961 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:17.249962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:17.250120 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:17.250180 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:17.250479 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 22:29:17.250583 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":32}
I20260808 22:29:17.251163 27644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:17.241570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:43267 (local address 127.0.0.1:38858)
0808 22:29:17.241671 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:17.241686 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:17.241769 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:17.242028 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:17.242031 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:17.242170 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:29:17.242276 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:17.242282 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.243300 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.243303 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:17.244147 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:17.244160 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.244719 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.244722 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:17.244821 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:17.245270 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:29:17.245282 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:29:17.246671 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:29:17.249204 (+ 2533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:17.249207 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:17.249209 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:17.249460 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:17.249768 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:17.249771 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:17.249773 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:17.249864 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:17.250237 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:29:17.250242 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:29:17.250329 (+ 87us) client_negotiation.cc:769] Sending connection context
0808 22:29:17.250869 (+ 540us) client_negotiation.cc:240] Negotiation successful
0808 22:29:17.250981 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31,"mutex_wait_us":16}
Aug 08 22:29:17 dist-test-slave-7f01 krb5kdc[27307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228156, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for kudu/127.5.11.252@KRBTEST.COM
I20260808 22:29:17.253006 27543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:17.242296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38650 (local address 127.5.11.253:42005)
0808 22:29:17.242412 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:17.242416 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:17.242428 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:29:17.242516 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:17.242518 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:17.242551 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:17.242608 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:17.243300 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.244081 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.245053 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.245270 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.245372 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:17.249193 (+ 3821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:17.249196 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:17.249198 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:17.249222 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:17.251151 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:17.251679 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:17.251682 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:17.251684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:17.251793 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:17.252096 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:17.252100 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:17.252102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:17.252322 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:17.252519 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:17.252734 (+ 215us) server_negotiation.cc:299] Negotiation successful
0808 22:29:17.252837 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 22:29:17.255088 27641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffe5072439ee40cb94b5cf7089ce3621: Bootstrap starting.
I20260808 22:29:17.256448 27430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:17.246596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:43267 (local address 127.0.0.1:38874)
0808 22:29:17.246690 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:17.246705 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:17.246816 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:17.247495 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:17.247498 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:17.247713 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:29:17.247848 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:17.247859 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.249434 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.249438 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:17.250191 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:17.250204 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.250831 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.250835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:17.250915 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:17.251370 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:29:17.251387 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:29:17.252818 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:29:17.254757 (+ 1939us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:17.254760 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:17.254762 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:17.255021 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:17.255361 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:17.255364 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:17.255366 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:17.255457 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:17.255921 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:29:17.255924 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:29:17.256029 (+ 105us) client_negotiation.cc:769] Sending connection context
0808 22:29:17.256146 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 22:29:17.256281 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 22:29:17.256489 27647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:17.246636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38874 (local address 127.5.11.252:43267)
0808 22:29:17.247267 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:17.247282 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:17.247296 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:29:17.247329 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:17.247332 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:17.247376 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:17.247478 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:17.247994 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.249304 (+ 1310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.250323 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.250677 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.251129 (+ 452us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:17.252966 (+ 1837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:17.252970 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:17.252972 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:17.252994 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:17.254631 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:17.255142 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:17.255145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:17.255146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:17.255255 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:17.255549 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:17.255554 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:17.255556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:17.255809 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:17.255910 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:17.256219 (+ 309us) server_negotiation.cc:299] Negotiation successful
0808 22:29:17.256320 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":128,"threads_started":1}
I20260808 22:29:17.258122 5167 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:29:17.258198 27641 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ffe5072439ee40cb94b5cf7089ce3621: Neither blocks nor log segments found. Creating new log.
Aug 08 22:29:17 dist-test-slave-7f01 krb5kdc[27307](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228156, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.253@KRBTEST.COM for kudu/127.5.11.252@KRBTEST.COM
I20260808 22:29:17.259157 27641 log.cc:824] T 00000000000000000000000000000000 P ffe5072439ee40cb94b5cf7089ce3621: Log is configured to *not* fsync() on all Append() calls
I20260808 22:29:17.260877 27426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f4328c9f9184ff4befe0245d945c3c1: Bootstrap starting.
I20260808 22:29:17.261405 27641 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffe5072439ee40cb94b5cf7089ce3621: No bootstrap required, opened a new log
I20260808 22:29:17.262432 27538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:17.248089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:43267 (local address 127.0.0.1:38880)
0808 22:29:17.248287 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:17.248300 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:17.248374 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:17.248734 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:17.248736 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:17.248911 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:29:17.248977 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:17.248982 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.250562 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.250566 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:17.251999 (+ 1433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:17.252010 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.256719 (+ 4709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.256723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:17.256807 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:17.257449 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:29:17.257462 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:29:17.258794 (+ 1332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:29:17.260844 (+ 2050us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:17.260847 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:17.260849 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:17.261109 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:17.261427 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:17.261431 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:17.261433 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:17.261532 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:17.261952 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:29:17.261956 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:29:17.262056 (+ 100us) client_negotiation.cc:769] Sending connection context
0808 22:29:17.262168 (+ 112us) client_negotiation.cc:240] Negotiation successful
0808 22:29:17.262294 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131}
I20260808 22:29:17.262513 27648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:17.248123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38880 (local address 127.5.11.252:43267)
0808 22:29:17.248537 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:17.248543 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:17.248552 (+ 9us) server_negotiation.cc:407] Connection header received
0808 22:29:17.248578 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:17.248580 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:17.248608 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:17.248716 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:17.249097 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.250408 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.256185 (+ 5777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.256513 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.256706 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:17.258942 (+ 2236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:17.258947 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:17.258951 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:17.258976 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:17.260718 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:17.261222 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:17.261226 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:17.261227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:17.261313 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:17.261639 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:17.261642 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:17.261644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:17.261840 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:17.261938 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:17.262245 (+ 307us) server_negotiation.cc:299] Negotiation successful
0808 22:29:17.262352 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":100,"threads_started":1}
I20260808 22:29:17.263674 27426 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f4328c9f9184ff4befe0245d945c3c1: Neither blocks nor log segments found. Creating new log.
I20260808 22:29:17.264501 27426 log.cc:824] T 00000000000000000000000000000000 P 9f4328c9f9184ff4befe0245d945c3c1: Log is configured to *not* fsync() on all Append() calls
I20260808 22:29:17.265360 27534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c69e70a4419e474aacdf3feadf9caf76: Bootstrap starting.
I20260808 22:29:17.265744 27641 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffe5072439ee40cb94b5cf7089ce3621 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4328c9f9184ff4befe0245d945c3c1" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 42125 } } peers { permanent_uuid: "c69e70a4419e474aacdf3feadf9caf76" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 42005 } } peers { permanent_uuid: "ffe5072439ee40cb94b5cf7089ce3621" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 43267 } }
I20260808 22:29:17.265956 27641 raft_consensus.cc:374] T 00000000000000000000000000000000 P ffe5072439ee40cb94b5cf7089ce3621 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:29:17.266474 27426 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f4328c9f9184ff4befe0245d945c3c1: No bootstrap required, opened a new log
I20260808 22:29:17.266561 27641 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffe5072439ee40cb94b5cf7089ce3621 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffe5072439ee40cb94b5cf7089ce3621, State: Initialized, Role: FOLLOWER
I20260808 22:29:17.267213 27641 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffe5072439ee40cb94b5cf7089ce3621 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4328c9f9184ff4befe0245d945c3c1" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 42125 } } peers { permanent_uuid: "c69e70a4419e474aacdf3feadf9caf76" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 42005 } } peers { permanent_uuid: "ffe5072439ee40cb94b5cf7089ce3621" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 43267 } }
I20260808 22:29:17.267726 27534 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c69e70a4419e474aacdf3feadf9caf76: Neither blocks nor log segments found. Creating new log.
I20260808 22:29:17.268563 27534 log.cc:824] T 00000000000000000000000000000000 P c69e70a4419e474aacdf3feadf9caf76: Log is configured to *not* fsync() on all Append() calls
I20260808 22:29:17.268760 27641 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffe5072439ee40cb94b5cf7089ce3621 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:29:17.270349 27426 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f4328c9f9184ff4befe0245d945c3c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4328c9f9184ff4befe0245d945c3c1" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 42125 } } peers { permanent_uuid: "c69e70a4419e474aacdf3feadf9caf76" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 42005 } } peers { permanent_uuid: "ffe5072439ee40cb94b5cf7089ce3621" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 43267 } }
I20260808 22:29:17.270908 27426 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f4328c9f9184ff4befe0245d945c3c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:29:17.270999 27426 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f4328c9f9184ff4befe0245d945c3c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f4328c9f9184ff4befe0245d945c3c1, State: Initialized, Role: FOLLOWER
I20260808 22:29:17.271471 27426 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f4328c9f9184ff4befe0245d945c3c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4328c9f9184ff4befe0245d945c3c1" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 42125 } } peers { permanent_uuid: "c69e70a4419e474aacdf3feadf9caf76" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 42005 } } peers { permanent_uuid: "ffe5072439ee40cb94b5cf7089ce3621" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 43267 } }
I20260808 22:29:17.270825 27667 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffe5072439ee40cb94b5cf7089ce3621 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4328c9f9184ff4befe0245d945c3c1" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 42125 } } peers { permanent_uuid: "c69e70a4419e474aacdf3feadf9caf76" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 42005 } } peers { permanent_uuid: "ffe5072439ee40cb94b5cf7089ce3621" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 43267 } } }
I20260808 22:29:17.273277 27667 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffe5072439ee40cb94b5cf7089ce3621 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:29:17.273624 27426 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f4328c9f9184ff4befe0245d945c3c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:29:17.272680 27544 master_runner.cc:186] Error getting master registration for 127.5.11.254:42125: OK, instance_id { permanent_uuid: "9f4328c9f9184ff4befe0245d945c3c1" instance_seqno: 1786228156782942 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 22:29:17.276309 27544 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 22:29:17.277613 27543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:17.266281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38652 (local address 127.5.11.253:42005)
0808 22:29:17.266645 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:17.266649 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:17.266664 (+ 15us) server_negotiation.cc:407] Connection header received
0808 22:29:17.266800 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:17.266804 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:17.266846 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:17.266915 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:17.267887 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.268555 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.269715 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.270078 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.270178 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:17.272209 (+ 2031us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:17.272214 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:17.272217 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:17.272244 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:17.274132 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:17.274626 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:17.274629 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:17.274631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:17.274712 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:17.275045 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:17.275050 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:17.275053 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:17.275309 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:17.275452 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:17.277297 (+ 1845us) server_negotiation.cc:299] Negotiation successful
0808 22:29:17.277444 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293}
I20260808 22:29:17.278519 27534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c69e70a4419e474aacdf3feadf9caf76: No bootstrap required, opened a new log
W20260808 22:29:17.279759 27358 tablet.cc:2369] T 00000000000000000000000000000000 P 9f4328c9f9184ff4befe0245d945c3c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:29:17.282325 27429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:17.271574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48446 (local address 127.5.11.254:42125)
0808 22:29:17.271704 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:17.271708 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:17.272864 (+ 1156us) server_negotiation.cc:407] Connection header received
0808 22:29:17.273047 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:17.273052 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:17.273104 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:17.273157 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:17.274070 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.274903 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.276338 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.276667 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.276756 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:17.278629 (+ 1873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:17.278633 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:17.278635 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:17.278661 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:17.280298 (+ 1637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:17.280818 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:17.280823 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:17.280826 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:17.280917 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:17.281237 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:17.281243 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:17.281247 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:17.281484 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:17.281620 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:17.282020 (+ 400us) server_negotiation.cc:299] Negotiation successful
0808 22:29:17.282156 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":7,"server-negotiator.queue_time_us":51}
I20260808 22:29:17.278419 27322 master_runner.cc:186] Error getting master registration for 127.5.11.253:42005: OK, instance_id { permanent_uuid: "c69e70a4419e474aacdf3feadf9caf76" instance_seqno: 1786228156993607 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 22:29:17.282517 27322 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 22:29:17.284370 27534 raft_consensus.cc:348] T 00000000000000000000000000000000 P c69e70a4419e474aacdf3feadf9caf76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4328c9f9184ff4befe0245d945c3c1" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 42125 } } peers { permanent_uuid: "c69e70a4419e474aacdf3feadf9caf76" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 42005 } } peers { permanent_uuid: "ffe5072439ee40cb94b5cf7089ce3621" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 43267 } }
I20260808 22:29:17.284586 27534 raft_consensus.cc:374] T 00000000000000000000000000000000 P c69e70a4419e474aacdf3feadf9caf76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:29:17.284650 27534 raft_consensus.cc:729] T 00000000000000000000000000000000 P c69e70a4419e474aacdf3feadf9caf76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c69e70a4419e474aacdf3feadf9caf76, State: Initialized, Role: FOLLOWER
I20260808 22:29:17.285135 27534 consensus_queue.cc:260] T 00000000000000000000000000000000 P c69e70a4419e474aacdf3feadf9caf76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4328c9f9184ff4befe0245d945c3c1" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 42125 } } peers { permanent_uuid: "c69e70a4419e474aacdf3feadf9caf76" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 42005 } } peers { permanent_uuid: "ffe5072439ee40cb94b5cf7089ce3621" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 43267 } }
I20260808 22:29:17.286145 27648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:17.272433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38886 (local address 127.5.11.252:43267)
0808 22:29:17.272541 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:17.272547 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:17.276060 (+ 3513us) server_negotiation.cc:407] Connection header received
0808 22:29:17.276214 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:17.276217 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:17.276266 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:17.276321 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:17.278457 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.279233 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.280427 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:17.280803 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:17.280885 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:17.282479 (+ 1594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:17.282485 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:17.282489 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:17.282540 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:17.284238 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:17.284740 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:17.284747 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:17.284751 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:17.284843 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:17.285186 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:17.285189 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:17.285190 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:17.285393 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:17.285525 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:17.285887 (+ 362us) server_negotiation.cc:299] Negotiation successful
0808 22:29:17.286010 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 22:29:17.286422 27686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f4328c9f9184ff4befe0245d945c3c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4328c9f9184ff4befe0245d945c3c1" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 42125 } } peers { permanent_uuid: "c69e70a4419e474aacdf3feadf9caf76" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 42005 } } peers { permanent_uuid: "ffe5072439ee40cb94b5cf7089ce3621" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 43267 } } }
I20260808 22:29:17.286624 27686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f4328c9f9184ff4befe0245d945c3c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:29:17.286628 27534 sys_catalog.cc:564] T 00000000000000000000000000000000 P c69e70a4419e474aacdf3feadf9caf76 [sys.catalog]: configured and running, proceeding with master startup.
W20260808 22:29:17.287490 27685 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ffe5072439ee40cb94b5cf7089ce3621: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:29:17.287600 27685 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260808 22:29:17.291450 27718 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c69e70a4419e474aacdf3feadf9caf76: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:29:17.291620 27718 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
W20260808 22:29:17.292558 27702 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9f4328c9f9184ff4befe0245d945c3c1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:29:17.292671 27702 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:29:17.293258 27704 sys_catalog.cc:455] T 00000000000000000000000000000000 P c69e70a4419e474aacdf3feadf9caf76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4328c9f9184ff4befe0245d945c3c1" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 42125 } } peers { permanent_uuid: "c69e70a4419e474aacdf3feadf9caf76" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 42005 } } peers { permanent_uuid: "ffe5072439ee40cb94b5cf7089ce3621" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 43267 } } }
I20260808 22:29:17.293431 27704 sys_catalog.cc:458] T 00000000000000000000000000000000 P c69e70a4419e474aacdf3feadf9caf76 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:29:17.293604 27667 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffe5072439ee40cb94b5cf7089ce3621 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:29:17.293779 27667 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffe5072439ee40cb94b5cf7089ce3621 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4328c9f9184ff4befe0245d945c3c1" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 42125 } } peers { permanent_uuid: "c69e70a4419e474aacdf3feadf9caf76" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 42005 } } peers { permanent_uuid: "ffe5072439ee40cb94b5cf7089ce3621" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 43267 } }
I20260808 22:29:17.294706 27667 leader_election.cc:288] T 00000000000000000000000000000000 P ffe5072439ee40cb94b5cf7089ce3621 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f4328c9f9184ff4befe0245d945c3c1 (127.5.11.254:42125), c69e70a4419e474aacdf3feadf9caf76 (127.5.11.253:42005)
I20260808 22:29:17.295727 27494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ffe5072439ee40cb94b5cf7089ce3621" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c69e70a4419e474aacdf3feadf9caf76" is_pre_election: true
I20260808 22:29:17.296049 27494 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c69e70a4419e474aacdf3feadf9caf76 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffe5072439ee40cb94b5cf7089ce3621 in term 0.
I20260808 22:29:17.296072 27391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ffe5072439ee40cb94b5cf7089ce3621" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f4328c9f9184ff4befe0245d945c3c1" is_pre_election: true
I20260808 22:29:17.296309 27391 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9f4328c9f9184ff4befe0245d945c3c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ffe5072439ee40cb94b5cf7089ce3621 in term 0.
I20260808 22:29:17.296595 27573 leader_election.cc:302] T 00000000000000000000000000000000 P ffe5072439ee40cb94b5cf7089ce3621 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c69e70a4419e474aacdf3feadf9caf76, ffe5072439ee40cb94b5cf7089ce3621; no voters:
I20260808 22:29:17.297058 27434 master_runner.cc:263] No leader master found from master c69e70a4419e474aacdf3feadf9caf76
I20260808 22:29:17.297122 27434 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 22:29:17.297098 27667 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffe5072439ee40cb94b5cf7089ce3621 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:29:17.297196 27667 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffe5072439ee40cb94b5cf7089ce3621 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:29:17.297255 27667 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffe5072439ee40cb94b5cf7089ce3621 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:29:17.298571 27667 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffe5072439ee40cb94b5cf7089ce3621 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4328c9f9184ff4befe0245d945c3c1" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 42125 } } peers { permanent_uuid: "c69e70a4419e474aacdf3feadf9caf76" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 42005 } } peers { permanent_uuid: "ffe5072439ee40cb94b5cf7089ce3621" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 43267 } }
I20260808 22:29:17.299154 27391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ffe5072439ee40cb94b5cf7089ce3621" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f4328c9f9184ff4befe0245d945c3c1"
I20260808 22:29:17.299268 27391 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f4328c9f9184ff4befe0245d945c3c1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:29:17.300613 27391 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 9f4328c9f9184ff4befe0245d945c3c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffe5072439ee40cb94b5cf7089ce3621 in term 1.
I20260808 22:29:17.300911 27575 leader_election.cc:302] T 00000000000000000000000000000000 P ffe5072439ee40cb94b5cf7089ce3621 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f4328c9f9184ff4befe0245d945c3c1, ffe5072439ee40cb94b5cf7089ce3621; no voters:
I20260808 22:29:17.301355 27667 leader_election.cc:288] T 00000000000000000000000000000000 P ffe5072439ee40cb94b5cf7089ce3621 [CANDIDATE]: Term 1 election: Requested vote from peers 9f4328c9f9184ff4befe0245d945c3c1 (127.5.11.254:42125), c69e70a4419e474aacdf3feadf9caf76 (127.5.11.253:42005)
I20260808 22:29:17.301463 27667 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffe5072439ee40cb94b5cf7089ce3621 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:29:17.301544 27667 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffe5072439ee40cb94b5cf7089ce3621 [term 1 LEADER]: Becoming Leader. State: Replica: ffe5072439ee40cb94b5cf7089ce3621, State: Running, Role: LEADER
I20260808 22:29:17.301842 27667 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffe5072439ee40cb94b5cf7089ce3621 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4328c9f9184ff4befe0245d945c3c1" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 42125 } } peers { permanent_uuid: "c69e70a4419e474aacdf3feadf9caf76" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 42005 } } peers { permanent_uuid: "ffe5072439ee40cb94b5cf7089ce3621" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 43267 } }
I20260808 22:29:17.303516 27667 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffe5072439ee40cb94b5cf7089ce3621 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffe5072439ee40cb94b5cf7089ce3621. Latest consensus state: current_term: 1 leader_uuid: "ffe5072439ee40cb94b5cf7089ce3621" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4328c9f9184ff4befe0245d945c3c1" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 42125 } } peers { permanent_uuid: "c69e70a4419e474aacdf3feadf9caf76" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 42005 } } peers { permanent_uuid: "ffe5072439ee40cb94b5cf7089ce3621" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 43267 } } }
I20260808 22:29:17.303617 27667 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffe5072439ee40cb94b5cf7089ce3621 [sys.catalog]: This master's current role is: LEADER
I20260808 22:29:17.304263 27494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "ffe5072439ee40cb94b5cf7089ce3621" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c69e70a4419e474aacdf3feadf9caf76"
I20260808 22:29:17.304380 27494 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c69e70a4419e474aacdf3feadf9caf76 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:29:17.305660 27494 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c69e70a4419e474aacdf3feadf9caf76 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ffe5072439ee40cb94b5cf7089ce3621 in term 1.
I20260808 22:29:17.309607 27726 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:29:17.311072 27726 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:29:17.316530 27494 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c69e70a4419e474aacdf3feadf9caf76 [term 1 FOLLOWER]: Refusing update from remote peer ffe5072439ee40cb94b5cf7089ce3621: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:29:17.317235 27724 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ffe5072439ee40cb94b5cf7089ce3621 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c69e70a4419e474aacdf3feadf9caf76" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 42005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:29:17.318689 27391 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9f4328c9f9184ff4befe0245d945c3c1 [term 1 FOLLOWER]: Refusing update from remote peer ffe5072439ee40cb94b5cf7089ce3621: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:29:17.319291 27724 consensus_queue.cc:1035] T 00000000000000000000000000000000 P ffe5072439ee40cb94b5cf7089ce3621 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f4328c9f9184ff4befe0245d945c3c1" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 42125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:29:17.322438 27704 sys_catalog.cc:455] T 00000000000000000000000000000000 P c69e70a4419e474aacdf3feadf9caf76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffe5072439ee40cb94b5cf7089ce3621. Latest consensus state: current_term: 1 leader_uuid: "ffe5072439ee40cb94b5cf7089ce3621" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4328c9f9184ff4befe0245d945c3c1" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 42125 } } peers { permanent_uuid: "c69e70a4419e474aacdf3feadf9caf76" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 42005 } } peers { permanent_uuid: "ffe5072439ee40cb94b5cf7089ce3621" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 43267 } } }
I20260808 22:29:17.322551 27704 sys_catalog.cc:458] T 00000000000000000000000000000000 P c69e70a4419e474aacdf3feadf9caf76 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:29:17.322579 27686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f4328c9f9184ff4befe0245d945c3c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffe5072439ee40cb94b5cf7089ce3621. Latest consensus state: current_term: 1 leader_uuid: "ffe5072439ee40cb94b5cf7089ce3621" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4328c9f9184ff4befe0245d945c3c1" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 42125 } } peers { permanent_uuid: "c69e70a4419e474aacdf3feadf9caf76" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 42005 } } peers { permanent_uuid: "ffe5072439ee40cb94b5cf7089ce3621" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 43267 } } }
I20260808 22:29:17.322691 27686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f4328c9f9184ff4befe0245d945c3c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:29:17.325274 27724 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffe5072439ee40cb94b5cf7089ce3621 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ffe5072439ee40cb94b5cf7089ce3621" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4328c9f9184ff4befe0245d945c3c1" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 42125 } } peers { permanent_uuid: "c69e70a4419e474aacdf3feadf9caf76" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 42005 } } peers { permanent_uuid: "ffe5072439ee40cb94b5cf7089ce3621" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 43267 } } }
I20260808 22:29:17.325393 27724 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffe5072439ee40cb94b5cf7089ce3621 [sys.catalog]: This master's current role is: LEADER
I20260808 22:29:17.326455 27686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f4328c9f9184ff4befe0245d945c3c1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ffe5072439ee40cb94b5cf7089ce3621" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4328c9f9184ff4befe0245d945c3c1" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 42125 } } peers { permanent_uuid: "c69e70a4419e474aacdf3feadf9caf76" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 42005 } } peers { permanent_uuid: "ffe5072439ee40cb94b5cf7089ce3621" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 43267 } } }
I20260808 22:29:17.326572 27686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f4328c9f9184ff4befe0245d945c3c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:29:17.327062 27727 mvcc.cc:205] Tried to move back new op lower bound from 7316390532361232384 to 7316390532311273472. Current Snapshot: MvccSnapshot[applied={T|T < 7316390532361232384}]
I20260808 22:29:17.329388 27730 mvcc.cc:205] Tried to move back new op lower bound from 7316390532361232384 to 7316390532311273472. Current Snapshot: MvccSnapshot[applied={T|T < 7316390532361232384}]
I20260808 22:29:17.331251 27726 catalog_manager.cc:1347] Generated new cluster ID: 48cd2f074525476baf8c6d0b75e92fa3
I20260808 22:29:17.331317 27726 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:29:17.338835 27667 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffe5072439ee40cb94b5cf7089ce3621 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "ffe5072439ee40cb94b5cf7089ce3621" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4328c9f9184ff4befe0245d945c3c1" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 42125 } } peers { permanent_uuid: "c69e70a4419e474aacdf3feadf9caf76" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 42005 } } peers { permanent_uuid: "ffe5072439ee40cb94b5cf7089ce3621" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 43267 } } }
I20260808 22:29:17.338948 27667 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffe5072439ee40cb94b5cf7089ce3621 [sys.catalog]: This master's current role is: LEADER
I20260808 22:29:17.339907 27704 sys_catalog.cc:455] T 00000000000000000000000000000000 P c69e70a4419e474aacdf3feadf9caf76 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "ffe5072439ee40cb94b5cf7089ce3621" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f4328c9f9184ff4befe0245d945c3c1" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 42125 } } peers { permanent_uuid: "c69e70a4419e474aacdf3feadf9caf76" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 42005 } } peers { permanent_uuid: "ffe5072439ee40cb94b5cf7089ce3621" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 43267 } } }
I20260808 22:29:17.340030 27704 sys_catalog.cc:458] T 00000000000000000000000000000000 P c69e70a4419e474aacdf3feadf9caf76 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:29:17.342406 27729 mvcc.cc:205] Tried to move back new op lower bound from 7316390532361232384 to 7316390532311273472. Current Snapshot: MvccSnapshot[applied={T|T < 7316390532361232384}]
I20260808 22:29:17.365202 27726 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:29:17.366465 27726 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:29:17.386891 27726 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ffe5072439ee40cb94b5cf7089ce3621: Generated new TSK 0
I20260808 22:29:17.387876 27726 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:29:18.292906 27718 catalog_manager.cc:1259] Loaded cluster ID: 48cd2f074525476baf8c6d0b75e92fa3
I20260808 22:29:18.293030 27718 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c69e70a4419e474aacdf3feadf9caf76: loading cluster ID for follower catalog manager: success
I20260808 22:29:18.293831 27702 catalog_manager.cc:1259] Loaded cluster ID: 48cd2f074525476baf8c6d0b75e92fa3
I20260808 22:29:18.293892 27702 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9f4328c9f9184ff4befe0245d945c3c1: loading cluster ID for follower catalog manager: success
I20260808 22:29:18.294772 27718 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c69e70a4419e474aacdf3feadf9caf76: acquiring CA information for follower catalog manager: success
I20260808 22:29:18.295472 27702 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9f4328c9f9184ff4befe0245d945c3c1: acquiring CA information for follower catalog manager: success
I20260808 22:29:18.296149 27718 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c69e70a4419e474aacdf3feadf9caf76: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 22:29:18.296754 27702 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9f4328c9f9184ff4befe0245d945c3c1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 22:29:18.450858 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 27322
I20260808 22:29:18.465691 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 27434
I20260808 22:29:18.476574 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 27544
2026-08-08T22:29:18Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3257 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:29:18 dist-test-slave-7f01 krb5kdc[27752](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:29:18 dist-test-slave-7f01 krb5kdc[27752](info): set up 2 sockets
Aug 08 22:29:18 dist-test-slave-7f01 krb5kdc[27752](info): commencing operation
krb5kdc: starting...
W20260808 22:29:19.739630 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.235s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:29:19 dist-test-slave-7f01 krb5kdc[27752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:29:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:29:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.11.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
I20260808 22:29:19.866304 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:44661
--webserver_interface=127.5.11.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:45907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:44661
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:29:19.982172 27773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:29:19.982427 27773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:29:19.982491 27773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:29:19.986408 27773 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:29:19.986461 27773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:29:19.986480 27773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:29:19.986544 27773 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:29:19.986572 27773 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:29:19.991714 27773 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:45907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:44661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:44661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:29:19.992938 27773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:29:19.993985 27773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:29:20.002434 27784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:20.002399 27782 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:29:20.002599 27773 server_base.cc:1034] running on GCE node
W20260808 22:29:20.002411 27788 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:29:20.003206 27773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:29:20.004372 27773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:29:20.005568 27773 hybrid_clock.cc:648] HybridClock initialized: now 1786228160005547 us; error 31 us; skew 500 ppm
Aug 08 22:29:20 dist-test-slave-7f01 krb5kdc[27752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228160, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:29:20.008348 27773 init.cc:377] Logged in from keytab as kudu/127.5.11.254@KRBTEST.COM (short username kudu)
I20260808 22:29:20.009210 27773 webserver.cc:460] Webserver started at http://127.5.11.254:45703/ using document root <none> and password file <none>
I20260808 22:29:20.009774 27773 fs_manager.cc:359] Metadata directory not provided
I20260808 22:29:20.009867 27773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:29:20.010110 27773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:29:20.012210 27773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "a3fda05258984db1ad263edd51817b13"
format_stamp: "Formatted at 2026-08-08 22:29:20 on dist-test-slave-7f01"
server_key: "9fedc328644bad5ed1bd32fd2cc0dd99"
server_key_iv: "642bf76527759a1caf7e36592a7470f4"
server_key_version: "encryptionkey@0"
I20260808 22:29:20.012830 27773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "a3fda05258984db1ad263edd51817b13"
format_stamp: "Formatted at 2026-08-08 22:29:20 on dist-test-slave-7f01"
server_key: "9fedc328644bad5ed1bd32fd2cc0dd99"
server_key_iv: "642bf76527759a1caf7e36592a7470f4"
server_key_version: "encryptionkey@0"
I20260808 22:29:20.016464 27773 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 22:29:20.018735 27796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:20.019990 27773 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:29:20.020130 27773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "a3fda05258984db1ad263edd51817b13"
format_stamp: "Formatted at 2026-08-08 22:29:20 on dist-test-slave-7f01"
server_key: "9fedc328644bad5ed1bd32fd2cc0dd99"
server_key_iv: "642bf76527759a1caf7e36592a7470f4"
server_key_version: "encryptionkey@0"
I20260808 22:29:20.020241 27773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:29:20.035089 27773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:29:20.037736 27773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:29:20.037945 27773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:29:20.045058 27773 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:44661
I20260808 22:29:20.045070 27872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:44661 every 8 connection(s)
I20260808 22:29:20.045993 27773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:29:20.048800 27874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:29:20.053020 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 27773
I20260808 22:29:20.053148 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:29:20.053467 5167 external_mini_cluster.cc:1428] Setting key b5c7e9024e618774fb9718d706eaf7b3
I20260808 22:29:20.054787 27874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3fda05258984db1ad263edd51817b13: Bootstrap starting.
I20260808 22:29:20.057245 27874 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3fda05258984db1ad263edd51817b13: Neither blocks nor log segments found. Creating new log.
I20260808 22:29:20.058048 27874 log.cc:824] T 00000000000000000000000000000000 P a3fda05258984db1ad263edd51817b13: Log is configured to *not* fsync() on all Append() calls
I20260808 22:29:20.059875 27874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3fda05258984db1ad263edd51817b13: No bootstrap required, opened a new log
Aug 08 22:29:20 dist-test-slave-7f01 krb5kdc[27752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:29:20.064718 27874 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3fda05258984db1ad263edd51817b13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3fda05258984db1ad263edd51817b13" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 44661 } }
I20260808 22:29:20.065013 27874 raft_consensus.cc:374] T 00000000000000000000000000000000 P a3fda05258984db1ad263edd51817b13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:29:20.065095 27874 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3fda05258984db1ad263edd51817b13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3fda05258984db1ad263edd51817b13, State: Initialized, Role: FOLLOWER
I20260808 22:29:20.065578 27874 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3fda05258984db1ad263edd51817b13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3fda05258984db1ad263edd51817b13" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 44661 } }
I20260808 22:29:20.065775 27874 raft_consensus.cc:388] T 00000000000000000000000000000000 P a3fda05258984db1ad263edd51817b13 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:29:20.065857 27874 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3fda05258984db1ad263edd51817b13 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:29:20.065963 27874 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3fda05258984db1ad263edd51817b13 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:29:20.067085 27874 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3fda05258984db1ad263edd51817b13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3fda05258984db1ad263edd51817b13" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 44661 } }
I20260808 22:29:20.067440 27874 leader_election.cc:302] T 00000000000000000000000000000000 P a3fda05258984db1ad263edd51817b13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3fda05258984db1ad263edd51817b13; no voters:
I20260808 22:29:20.067807 27874 leader_election.cc:288] T 00000000000000000000000000000000 P a3fda05258984db1ad263edd51817b13 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:29:20.067898 27881 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3fda05258984db1ad263edd51817b13 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:29:20.068135 27881 raft_consensus.cc:686] T 00000000000000000000000000000000 P a3fda05258984db1ad263edd51817b13 [term 1 LEADER]: Becoming Leader. State: Replica: a3fda05258984db1ad263edd51817b13, State: Running, Role: LEADER
I20260808 22:29:20.068230 27878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:20.054769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36532 (local address 127.5.11.254:44661)
0808 22:29:20.055236 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:20.055252 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:20.055287 (+ 35us) server_negotiation.cc:407] Connection header received
0808 22:29:20.056009 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:20.056028 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:20.056349 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:20.056626 (+ 277us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:20.057559 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:20.058567 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:20.059416 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:20.059947 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:20.060116 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:20.062876 (+ 2760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:20.062885 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:20.062905 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:20.062944 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:20.064957 (+ 2013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:20.065463 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:20.065469 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:20.065474 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:20.065617 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:20.065975 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:20.065979 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:20.065982 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:20.066496 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:20.066691 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:20.066938 (+ 247us) server_negotiation.cc:299] Negotiation successful
0808 22:29:20.067245 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":157,"threads_started":1}
I20260808 22:29:20.068496 27881 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3fda05258984db1ad263edd51817b13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3fda05258984db1ad263edd51817b13" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 44661 } }
I20260808 22:29:20.069083 27874 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3fda05258984db1ad263edd51817b13 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:29:20.070204 27883 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3fda05258984db1ad263edd51817b13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3fda05258984db1ad263edd51817b13. Latest consensus state: current_term: 1 leader_uuid: "a3fda05258984db1ad263edd51817b13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3fda05258984db1ad263edd51817b13" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 44661 } } }
I20260808 22:29:20.070207 27882 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3fda05258984db1ad263edd51817b13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a3fda05258984db1ad263edd51817b13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3fda05258984db1ad263edd51817b13" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 44661 } } }
I20260808 22:29:20.070364 27882 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3fda05258984db1ad263edd51817b13 [sys.catalog]: This master's current role is: LEADER
I20260808 22:29:20.070364 27883 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3fda05258984db1ad263edd51817b13 [sys.catalog]: This master's current role is: LEADER
I20260808 22:29:20.071552 5167 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260808 22:29:20.074553 27907 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a3fda05258984db1ad263edd51817b13: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:29:20.074645 27907 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:29:20.074723 27895 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:29:20.075445 27895 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:29:20.080729 27895 catalog_manager.cc:1347] Generated new cluster ID: 19b3be4455cd4e5f8b1c5e05c22121c0
I20260808 22:29:20.080791 27895 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:29:20.091599 27895 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:29:20.092486 27895 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:29:20.104768 27895 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a3fda05258984db1ad263edd51817b13: Generated new TSK 0
I20260808 22:29:20.105398 27895 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:29:20.127758 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 27773
2026-08-08T22:29:20Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1654 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:29:20 dist-test-slave-7f01 krb5kdc[27921](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:29:20 dist-test-slave-7f01 krb5kdc[27921](info): set up 2 sockets
Aug 08 22:29:20 dist-test-slave-7f01 krb5kdc[27921](info): commencing operation
krb5kdc: starting...
W20260808 22:29:21.410560 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.252s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:29:21 dist-test-slave-7f01 krb5kdc[27921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:29:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:29:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.11.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
I20260808 22:29:21.538180 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:46669
--webserver_interface=127.5.11.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:34707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:46669
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:29:21.650930 27938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:29:21.651185 27938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:29:21.651224 27938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:29:21.655047 27938 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:29:21.655094 27938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:29:21.655113 27938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:29:21.655129 27938 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:29:21.655143 27938 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:29:21.660295 27938 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:34707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:46669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:46669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:29:21.661211 27938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:29:21.662154 27938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:29:21.670533 27947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:21.670550 27946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:21.670540 27949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:29:21.670856 27938 server_base.cc:1034] running on GCE node
I20260808 22:29:21.671569 27938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:29:21.672770 27938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:29:21.673931 27938 hybrid_clock.cc:648] HybridClock initialized: now 1786228161673915 us; error 29 us; skew 500 ppm
Aug 08 22:29:21 dist-test-slave-7f01 krb5kdc[27921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228161, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:29:21.676635 27938 init.cc:377] Logged in from keytab as kudu/127.5.11.254@KRBTEST.COM (short username kudu)
I20260808 22:29:21.677464 27938 webserver.cc:460] Webserver started at http://127.5.11.254:36703/ using document root <none> and password file <none>
I20260808 22:29:21.678023 27938 fs_manager.cc:359] Metadata directory not provided
I20260808 22:29:21.678117 27938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:29:21.678347 27938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:29:21.680409 27938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "ff42823b2ab54e03bac154722af86662"
format_stamp: "Formatted at 2026-08-08 22:29:21 on dist-test-slave-7f01"
server_key: "4fe00a98e8959ff4aae4e018e4c16ce3"
server_key_iv: "bbe40d0c30c37a9cd9f824e048f96b72"
server_key_version: "encryptionkey@0"
I20260808 22:29:21.680984 27938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "ff42823b2ab54e03bac154722af86662"
format_stamp: "Formatted at 2026-08-08 22:29:21 on dist-test-slave-7f01"
server_key: "4fe00a98e8959ff4aae4e018e4c16ce3"
server_key_iv: "bbe40d0c30c37a9cd9f824e048f96b72"
server_key_version: "encryptionkey@0"
I20260808 22:29:21.684594 27938 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:29:21.687001 27961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:21.688207 27938 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:29:21.688344 27938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "ff42823b2ab54e03bac154722af86662"
format_stamp: "Formatted at 2026-08-08 22:29:21 on dist-test-slave-7f01"
server_key: "4fe00a98e8959ff4aae4e018e4c16ce3"
server_key_iv: "bbe40d0c30c37a9cd9f824e048f96b72"
server_key_version: "encryptionkey@0"
I20260808 22:29:21.688452 27938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:29:21.715992 27938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:29:21.718681 27938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:29:21.718887 27938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:29:21.725970 27938 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:46669
I20260808 22:29:21.726253 28045 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:46669 every 8 connection(s)
I20260808 22:29:21.726945 27938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:29:21.729691 28046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:29:21.734731 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 27938
I20260808 22:29:21.734874 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:29:21.735172 5167 external_mini_cluster.cc:1428] Setting key 65ca20b2c2bfb5de80ceca32ceeb46c9
I20260808 22:29:21.735458 28046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662: Bootstrap starting.
I20260808 22:29:21.737797 28046 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662: Neither blocks nor log segments found. Creating new log.
I20260808 22:29:21.738682 28046 log.cc:824] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662: Log is configured to *not* fsync() on all Append() calls
I20260808 22:29:21.740594 28046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662: No bootstrap required, opened a new log
Aug 08 22:29:21 dist-test-slave-7f01 krb5kdc[27921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:29:21.744599 28046 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff42823b2ab54e03bac154722af86662" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 46669 } }
I20260808 22:29:21.744813 28046 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:29:21.744885 28046 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff42823b2ab54e03bac154722af86662, State: Initialized, Role: FOLLOWER
I20260808 22:29:21.745332 28046 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff42823b2ab54e03bac154722af86662" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 46669 } }
I20260808 22:29:21.745502 28046 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:29:21.745589 28046 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:29:21.745702 28046 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:29:21.746520 28046 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff42823b2ab54e03bac154722af86662" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 46669 } }
I20260808 22:29:21.746917 28046 leader_election.cc:302] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff42823b2ab54e03bac154722af86662; no voters:
I20260808 22:29:21.747301 28046 leader_election.cc:288] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:29:21.747401 28052 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:29:21.747666 28052 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [term 1 LEADER]: Becoming Leader. State: Replica: ff42823b2ab54e03bac154722af86662, State: Running, Role: LEADER
I20260808 22:29:21.748042 28052 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff42823b2ab54e03bac154722af86662" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 46669 } }
I20260808 22:29:21.748670 28046 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:29:21.749701 28055 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff42823b2ab54e03bac154722af86662" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff42823b2ab54e03bac154722af86662" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 46669 } } }
I20260808 22:29:21.749751 28050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:21.736604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39312 (local address 127.5.11.254:46669)
0808 22:29:21.737012 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:21.737032 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:21.737065 (+ 33us) server_negotiation.cc:407] Connection header received
0808 22:29:21.737653 (+ 588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:21.737668 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:21.738033 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:21.738297 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:21.739260 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:21.740285 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:21.741168 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:21.741653 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:21.741824 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:21.744538 (+ 2714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:21.744543 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:21.744555 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:21.744581 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:21.746414 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:21.746905 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:21.746910 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:21.746914 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:21.747068 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:21.747444 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:21.747447 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:21.747449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:21.748005 (+ 556us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:21.748211 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:21.748478 (+ 267us) server_negotiation.cc:299] Negotiation successful
0808 22:29:21.748766 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":139,"threads_started":1}
I20260808 22:29:21.749845 28055 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [sys.catalog]: This master's current role is: LEADER
I20260808 22:29:21.750535 28056 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff42823b2ab54e03bac154722af86662. Latest consensus state: current_term: 1 leader_uuid: "ff42823b2ab54e03bac154722af86662" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff42823b2ab54e03bac154722af86662" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 46669 } } }
I20260808 22:29:21.750656 28056 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [sys.catalog]: This master's current role is: LEADER
I20260808 22:29:21.751359 28067 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:29:21.754529 28067 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:29:21.759509 28067 catalog_manager.cc:1347] Generated new cluster ID: 1bffbea253484f77945a73d2ea733db3
I20260808 22:29:21.759568 28067 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:29:21.773228 28067 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:29:21.774423 28067 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:29:21.785375 28067 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662: Generated new TSK 0
I20260808 22:29:21.786154 28067 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.11.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab.
Entry for principal kudu/127.5.11.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab.
I20260808 22:29:21.818568 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.193:0
--local_ip_for_outbound_sockets=127.5.11.193
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:46669
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:34707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:29:21.936174 28086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:29:21.936414 28086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:29:21.936477 28086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:29:21.940429 28086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:29:21.940481 28086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:29:21.940568 28086 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.193
I20260808 22:29:21.945730 28086 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:34707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:46669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.193
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:29:21.946733 28086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:29:21.947781 28086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:29:21.957412 28097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:21.957381 28094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:21.957374 28102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:29:21.957820 28086 server_base.cc:1034] running on GCE node
I20260808 22:29:21.958241 28086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:29:21.958801 28086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:29:21.960000 28086 hybrid_clock.cc:648] HybridClock initialized: now 1786228161959968 us; error 42 us; skew 500 ppm
Aug 08 22:29:21 dist-test-slave-7f01 krb5kdc[27921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228161, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:29:21.962734 28086 init.cc:377] Logged in from keytab as kudu/127.5.11.193@KRBTEST.COM (short username kudu)
I20260808 22:29:21.963605 28086 webserver.cc:460] Webserver started at http://127.5.11.193:40453/ using document root <none> and password file <none>
I20260808 22:29:21.964205 28086 fs_manager.cc:359] Metadata directory not provided
I20260808 22:29:21.964268 28086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:29:21.964524 28086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:29:21.966495 28086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/data/instance:
uuid: "d84376118b0d44df94afa76d495ae676"
format_stamp: "Formatted at 2026-08-08 22:29:21 on dist-test-slave-7f01"
server_key: "93dc6161b588eb8ca7158c229d8fa5a9"
server_key_iv: "e90ee9ab882aac8153fdc2e267b83b5b"
server_key_version: "encryptionkey@0"
I20260808 22:29:21.967077 28086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance:
uuid: "d84376118b0d44df94afa76d495ae676"
format_stamp: "Formatted at 2026-08-08 22:29:21 on dist-test-slave-7f01"
server_key: "93dc6161b588eb8ca7158c229d8fa5a9"
server_key_iv: "e90ee9ab882aac8153fdc2e267b83b5b"
server_key_version: "encryptionkey@0"
I20260808 22:29:21.970929 28086 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 22:29:21.973273 28115 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:21.974445 28086 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:29:21.974582 28086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/wal
uuid: "d84376118b0d44df94afa76d495ae676"
format_stamp: "Formatted at 2026-08-08 22:29:21 on dist-test-slave-7f01"
server_key: "93dc6161b588eb8ca7158c229d8fa5a9"
server_key_iv: "e90ee9ab882aac8153fdc2e267b83b5b"
server_key_version: "encryptionkey@0"
I20260808 22:29:21.974694 28086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:29:21.986177 28086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:29:21.988957 28086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:29:21.989140 28086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:29:21.989686 28086 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:29:21.990461 28086 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:29:21.990532 28086 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:21.990602 28086 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:29:21.990656 28086 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:21.999481 28086 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.193:43769
I20260808 22:29:21.999560 28278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.193:43769 every 8 connection(s)
I20260808 22:29:22.000360 28086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
I20260808 22:29:22.005473 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 28086
I20260808 22:29:22.005604 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance
I20260808 22:29:22.005901 5167 external_mini_cluster.cc:1428] Setting key b9f64b4b9fa2c1a68d3fa608b7a58f83
Aug 08 22:29:22 dist-test-slave-7f01 krb5kdc[27921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228161, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:29:22.014557 28050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:22.002085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:56677 (local address 127.5.11.254:46669)
0808 22:29:22.002229 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:22.002235 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:22.003269 (+ 1034us) server_negotiation.cc:407] Connection header received
0808 22:29:22.004179 (+ 910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:22.004181 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:22.004226 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:22.004298 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:22.005416 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:22.006210 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:22.007767 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:22.008173 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:22.008297 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:22.010677 (+ 2380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:22.010680 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:22.010682 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:22.010705 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:22.012287 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:22.012945 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:22.012948 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:22.012949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:22.013028 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:22.013430 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:22.013433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:22.013434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:22.013600 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:22.013677 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:22.014322 (+ 645us) server_negotiation.cc:299] Negotiation successful
0808 22:29:22.014431 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 22:29:22.015681 28285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:22.002496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.193:56677)
0808 22:29:22.003118 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:22.003157 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:22.003996 (+ 839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:22.004461 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:22.004467 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:22.004795 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:29:22.005259 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:22.005272 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:22.006363 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:22.006368 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:22.007610 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:22.007621 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:22.008314 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:22.008317 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:22.008451 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:22.009006 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:29:22.009022 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:29:22.010509 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:29:22.012408 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:22.012416 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:22.012441 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:22.012830 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:22.013120 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:22.013123 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:22.013126 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:22.013325 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:22.013716 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:29:22.013726 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:29:22.014054 (+ 328us) client_negotiation.cc:769] Sending connection context
0808 22:29:22.014279 (+ 225us) client_negotiation.cc:240] Negotiation successful
0808 22:29:22.014651 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":109,"threads_started":1}
I20260808 22:29:22.016944 28283 heartbeater.cc:347] Connected to a master server at 127.5.11.254:46669
I20260808 22:29:22.017177 28283 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.5.11.194@KRBTEST.COM; defaulting to no policy
I20260808 22:29:22.017783 28283 heartbeater.cc:511] Master 127.5.11.254:46669 requested a full tablet report, sending...
I20260808 22:29:22.019289 27983 ts_manager.cc:194] Registered new tserver with Master: d84376118b0d44df94afa76d495ae676 (127.5.11.193:43769)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.194@KRBTEST.COM" created.
I20260808 22:29:22.020753 27983 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.193@KRBTEST.COM'} at 127.5.11.193:56677
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab.
Entry for principal kudu/127.5.11.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab.
I20260808 22:29:22.036346 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.194:0
--local_ip_for_outbound_sockets=127.5.11.194
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:46669
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:34707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:29:22.146236 28288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:29:22.146490 28288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:29:22.146564 28288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:29:22.150661 28288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:29:22.150731 28288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:29:22.150811 28288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.194
I20260808 22:29:22.155717 28288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:34707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:46669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.194
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:29:22.156656 28288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:29:22.157565 28288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:29:22.166901 28296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:29:22.166992 28288 server_base.cc:1034] running on GCE node
W20260808 22:29:22.166909 28294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:22.166939 28293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:29:22.167555 28288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:29:22.168206 28288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:29:22.169394 28288 hybrid_clock.cc:648] HybridClock initialized: now 1786228162169384 us; error 30 us; skew 500 ppm
Aug 08 22:29:22 dist-test-slave-7f01 krb5kdc[27921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228162, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:29:22.172140 28288 init.cc:377] Logged in from keytab as kudu/127.5.11.194@KRBTEST.COM (short username kudu)
I20260808 22:29:22.172931 28288 webserver.cc:460] Webserver started at http://127.5.11.194:37437/ using document root <none> and password file <none>
I20260808 22:29:22.173431 28288 fs_manager.cc:359] Metadata directory not provided
I20260808 22:29:22.173491 28288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:29:22.173645 28288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:29:22.175493 28288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/data/instance:
uuid: "27db2e9af5d04eb2a695be5342e01774"
format_stamp: "Formatted at 2026-08-08 22:29:22 on dist-test-slave-7f01"
server_key: "52dedd1382beac0ef83ce7a07dfe95bf"
server_key_iv: "eed92f487c2011f7c09b785ca8f5538b"
server_key_version: "encryptionkey@0"
I20260808 22:29:22.176057 28288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance:
uuid: "27db2e9af5d04eb2a695be5342e01774"
format_stamp: "Formatted at 2026-08-08 22:29:22 on dist-test-slave-7f01"
server_key: "52dedd1382beac0ef83ce7a07dfe95bf"
server_key_iv: "eed92f487c2011f7c09b785ca8f5538b"
server_key_version: "encryptionkey@0"
I20260808 22:29:22.179656 28288 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 22:29:22.181960 28312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:22.183223 28288 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:29:22.183378 28288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/wal
uuid: "27db2e9af5d04eb2a695be5342e01774"
format_stamp: "Formatted at 2026-08-08 22:29:22 on dist-test-slave-7f01"
server_key: "52dedd1382beac0ef83ce7a07dfe95bf"
server_key_iv: "eed92f487c2011f7c09b785ca8f5538b"
server_key_version: "encryptionkey@0"
I20260808 22:29:22.183470 28288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:29:22.231662 28288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:29:22.235236 28288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:29:22.235406 28288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:29:22.235997 28288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:29:22.236821 28288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:29:22.236871 28288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:22.236912 28288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:29:22.236931 28288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:22.245554 28288 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.194:34481
I20260808 22:29:22.245580 28494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.194:34481 every 8 connection(s)
I20260808 22:29:22.246436 28288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
I20260808 22:29:22.253451 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 28288
I20260808 22:29:22.253583 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance
I20260808 22:29:22.253882 5167 external_mini_cluster.cc:1428] Setting key 78f4f739a8948624d216cd8a57d4bf95
Aug 08 22:29:22 dist-test-slave-7f01 krb5kdc[27921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228162, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:29:22.261161 28050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:22.248258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:59831 (local address 127.5.11.254:46669)
0808 22:29:22.248410 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:22.248414 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:22.249597 (+ 1183us) server_negotiation.cc:407] Connection header received
0808 22:29:22.250484 (+ 887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:22.250487 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:22.250571 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:22.250666 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:22.252057 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:22.252579 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:22.253927 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:22.254181 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:22.254262 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:22.257372 (+ 3110us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:22.257375 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:22.257377 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:22.257401 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:22.259100 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:22.259609 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:22.259613 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:22.259615 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:22.259705 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:22.260095 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:22.260100 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:22.260101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:22.260284 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:22.260395 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:22.260856 (+ 461us) server_negotiation.cc:299] Negotiation successful
0808 22:29:22.260993 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 22:29:22.261879 28497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:22.248644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.194:59831)
0808 22:29:22.249434 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:22.249501 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:22.250262 (+ 761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:22.250880 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:22.250891 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:22.251327 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:29:22.251912 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:22.251928 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:22.252684 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:22.252687 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:22.253805 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:22.253819 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:22.254483 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:22.254486 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:22.254693 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:22.255358 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:29:22.255380 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:29:22.257210 (+ 1830us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:29:22.259223 (+ 2013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:22.259228 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:22.259239 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:22.259498 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:22.259847 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:22.259851 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:22.259855 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:22.259993 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:22.260393 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:29:22.260400 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:29:22.260627 (+ 227us) client_negotiation.cc:769] Sending connection context
0808 22:29:22.260811 (+ 184us) client_negotiation.cc:240] Negotiation successful
0808 22:29:22.261111 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":148,"threads_started":1}
I20260808 22:29:22.262981 28495 heartbeater.cc:347] Connected to a master server at 127.5.11.254:46669
I20260808 22:29:22.263221 28495 heartbeater.cc:464] Registering TS with master...
I20260808 22:29:22.263677 28495 heartbeater.cc:511] Master 127.5.11.254:46669 requested a full tablet report, sending...
I20260808 22:29:22.264775 27983 ts_manager.cc:194] Registered new tserver with Master: 27db2e9af5d04eb2a695be5342e01774 (127.5.11.194:34481)
WARNING: no policy specified for kudu/127.5.11.195@KRBTEST.COM; defaulting to no policy
I20260808 22:29:22.265589 27983 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.194@KRBTEST.COM'} at 127.5.11.194:59831
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab.
Entry for principal kudu/127.5.11.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab.
I20260808 22:29:22.283166 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.195:0
--local_ip_for_outbound_sockets=127.5.11.195
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:46669
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:34707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:29:22.397621 28505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:29:22.397864 28505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:29:22.397904 28505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:29:22.401887 28505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:29:22.401944 28505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:29:22.402002 28505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.195
I20260808 22:29:22.407040 28505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:34707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:46669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.195
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:29:22.408034 28505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:29:22.409080 28505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:29:22.418708 28515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:22.418709 28512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:22.418682 28511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:29:22.419003 28505 server_base.cc:1034] running on GCE node
I20260808 22:29:22.419390 28505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:29:22.420071 28505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:29:22.421258 28505 hybrid_clock.cc:648] HybridClock initialized: now 1786228162421240 us; error 27 us; skew 500 ppm
Aug 08 22:29:22 dist-test-slave-7f01 krb5kdc[27921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228162, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:29:22.424003 28505 init.cc:377] Logged in from keytab as kudu/127.5.11.195@KRBTEST.COM (short username kudu)
I20260808 22:29:22.424875 28505 webserver.cc:460] Webserver started at http://127.5.11.195:43573/ using document root <none> and password file <none>
I20260808 22:29:22.425436 28505 fs_manager.cc:359] Metadata directory not provided
I20260808 22:29:22.425534 28505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:29:22.425765 28505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:29:22.427839 28505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/data/instance:
uuid: "e49df4478baa46cbaf98a2b1c1332060"
format_stamp: "Formatted at 2026-08-08 22:29:22 on dist-test-slave-7f01"
server_key: "b5d4f5261b0ab3ff58fe0ef707124d37"
server_key_iv: "aeb17ba00f9f4fd3223a8e3da3bfe8bc"
server_key_version: "encryptionkey@0"
I20260808 22:29:22.428447 28505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance:
uuid: "e49df4478baa46cbaf98a2b1c1332060"
format_stamp: "Formatted at 2026-08-08 22:29:22 on dist-test-slave-7f01"
server_key: "b5d4f5261b0ab3ff58fe0ef707124d37"
server_key_iv: "aeb17ba00f9f4fd3223a8e3da3bfe8bc"
server_key_version: "encryptionkey@0"
I20260808 22:29:22.432138 28505 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 22:29:22.434616 28525 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:22.435864 28505 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:29:22.435989 28505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/wal
uuid: "e49df4478baa46cbaf98a2b1c1332060"
format_stamp: "Formatted at 2026-08-08 22:29:22 on dist-test-slave-7f01"
server_key: "b5d4f5261b0ab3ff58fe0ef707124d37"
server_key_iv: "aeb17ba00f9f4fd3223a8e3da3bfe8bc"
server_key_version: "encryptionkey@0"
I20260808 22:29:22.436117 28505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:29:22.458232 28505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:29:22.461242 28505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:29:22.461447 28505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:29:22.462024 28505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:29:22.462847 28505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:29:22.462918 28505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:22.462998 28505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:29:22.463042 28505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:22.472409 28505 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.195:36615
I20260808 22:29:22.472445 28704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.195:36615 every 8 connection(s)
I20260808 22:29:22.473294 28505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
I20260808 22:29:22.479976 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 28505
I20260808 22:29:22.480122 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance
I20260808 22:29:22.480428 5167 external_mini_cluster.cc:1428] Setting key 9ffedf0c312099d572d424dd2d38671d
Aug 08 22:29:22 dist-test-slave-7f01 krb5kdc[27921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228162, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:29:22.486618 28050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:22.475105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:56777 (local address 127.5.11.254:46669)
0808 22:29:22.475239 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:22.475243 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:22.476608 (+ 1365us) server_negotiation.cc:407] Connection header received
0808 22:29:22.477242 (+ 634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:22.477245 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:22.477294 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:22.477373 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:22.478596 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:22.479156 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:22.480465 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:22.480785 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:22.480879 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:22.483462 (+ 2583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:22.483466 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:22.483468 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:22.483499 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:22.484790 (+ 1291us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:22.485247 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:22.485249 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:22.485250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:22.485330 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:22.485618 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:22.485620 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:22.485621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:22.485789 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:22.485864 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:22.486303 (+ 439us) server_negotiation.cc:299] Negotiation successful
0808 22:29:22.486399 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 22:29:22.487460 28709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:22.475483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.195:56777)
0808 22:29:22.476207 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:22.476247 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:22.477056 (+ 809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:22.477536 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:22.477543 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:22.477908 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:29:22.478457 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:22.478471 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:22.479283 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:22.479288 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:22.480342 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:22.480352 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:22.480947 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:22.480952 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:22.481121 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:22.481774 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:29:22.481796 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:29:22.483317 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:29:22.484902 (+ 1585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:22.484907 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:22.484918 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:22.485161 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:22.485408 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:22.485410 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:22.485412 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:22.485539 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:22.485891 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:29:22.485895 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:29:22.486105 (+ 210us) client_negotiation.cc:769] Sending connection context
0808 22:29:22.486255 (+ 150us) client_negotiation.cc:240] Negotiation successful
0808 22:29:22.486637 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":118,"threads_started":1}
I20260808 22:29:22.488694 28705 heartbeater.cc:347] Connected to a master server at 127.5.11.254:46669
I20260808 22:29:22.488942 28705 heartbeater.cc:464] Registering TS with master...
I20260808 22:29:22.489494 28705 heartbeater.cc:511] Master 127.5.11.254:46669 requested a full tablet report, sending...
I20260808 22:29:22.490708 27983 ts_manager.cc:194] Registered new tserver with Master: e49df4478baa46cbaf98a2b1c1332060 (127.5.11.195:36615)
I20260808 22:29:22.491485 27983 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.195@KRBTEST.COM'} at 127.5.11.195:56777
I20260808 22:29:22.494395 5167 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:29:22.503340 28050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:22.496406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39322 (local address 127.5.11.254:46669)
0808 22:29:22.496545 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:22.496549 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:22.496691 (+ 142us) server_negotiation.cc:407] Connection header received
0808 22:29:22.496784 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:22.496786 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:22.496824 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:22.496888 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:22.497575 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:22.498063 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:22.498821 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:22.499027 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:22.499100 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:22.500461 (+ 1361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:22.500465 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:22.500467 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:22.500485 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:22.501848 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:22.502226 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:22.502228 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:22.502229 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:22.502302 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:22.502533 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:22.502535 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:22.502536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:22.502730 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:22.502832 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:22.503036 (+ 204us) server_negotiation.cc:299] Negotiation successful
0808 22:29:22.503132 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 22:29:22.506075 27983 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39322:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:29:22.509173 27983 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:29:22.524039 28725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:22.517643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45402 (local address 127.5.11.194:34481)
0808 22:29:22.518466 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:22.518476 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:22.518527 (+ 51us) server_negotiation.cc:407] Connection header received
0808 22:29:22.518609 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:22.518614 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:22.518829 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:22.519042 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:29:22.519393 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:22.520678 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:22.522277 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:22.523111 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:22.523236 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:22.523398 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:22.523775 (+ 377us) server_negotiation.cc:299] Negotiation successful
0808 22:29:22.523877 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":726,"thread_start_us":103,"threads_started":1}
I20260808 22:29:22.524075 28727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:22.517848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:43769 (local address 127.0.0.1:47640)
0808 22:29:22.518181 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:22.518195 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:22.518268 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:22.519303 (+ 1035us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:22.519307 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:22.519329 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:29:22.519410 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:22.519416 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:22.520851 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:22.520854 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:22.522307 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:22.522318 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:22.523276 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:22.523279 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:22.523495 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:22.523585 (+ 90us) client_negotiation.cc:769] Sending connection context
0808 22:29:22.523702 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 22:29:22.523895 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":218,"thread_start_us":111,"threads_started":1}
I20260808 22:29:22.524075 28724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:22.517465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:34481 (local address 127.0.0.1:45402)
0808 22:29:22.517973 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:22.518027 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:22.518156 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:22.519053 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:22.519062 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:22.519093 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:29:22.519239 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:22.519248 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:22.520809 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:22.520812 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:22.522129 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:22.522140 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:22.523265 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:22.523269 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:22.523491 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:22.523585 (+ 94us) client_negotiation.cc:769] Sending connection context
0808 22:29:22.523702 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 22:29:22.523906 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"mutex_wait_us":167,"thread_start_us":95,"threads_started":1}
I20260808 22:29:22.524541 28729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:22.518158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47640 (local address 127.5.11.193:43769)
0808 22:29:22.518650 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:22.518656 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:22.518675 (+ 19us) server_negotiation.cc:407] Connection header received
0808 22:29:22.518738 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:22.518767 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:22.519000 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:22.519152 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:29:22.519540 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:22.520678 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:22.522436 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:22.523175 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:22.523274 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:22.523397 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:22.524261 (+ 864us) server_negotiation.cc:299] Negotiation successful
0808 22:29:22.524361 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":62,"threads_started":1}
I20260808 22:29:22.524893 28730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:22.518197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:36615 (local address 127.0.0.1:51098)
0808 22:29:22.519320 (+ 1123us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:22.519334 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:22.519424 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:22.519923 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:22.519925 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:22.519944 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:29:22.520053 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:22.520062 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:22.521151 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:22.521154 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:22.522306 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:22.522318 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:22.524502 (+ 2184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:22.524508 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:22.524600 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:22.524629 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 22:29:22.524692 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 22:29:22.524771 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1054,"mutex_wait_us":8,"thread_start_us":86,"threads_started":1}
I20260808 22:29:22.525231 28731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:22.518612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51098 (local address 127.5.11.195:36615)
0808 22:29:22.518931 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:22.518943 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:22.519529 (+ 586us) server_negotiation.cc:407] Connection header received
0808 22:29:22.519607 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:22.519611 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:22.519778 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:22.519945 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:29:22.520179 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:22.521006 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:22.523460 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:22.524378 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:22.524656 (+ 278us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:22.524847 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:22.524902 (+ 55us) server_negotiation.cc:299] Negotiation successful
0808 22:29:22.525070 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":110,"threads_started":1}
I20260808 22:29:22.527619 28603 tablet_service.cc:1467] Processing CreateTablet for tablet 7f0d77340b8340b78d05302c7f2d4f87 (DEFAULT_TABLE table=test-table [id=aae363461a284482bd041c8cbb5c0619]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:29:22.527684 28177 tablet_service.cc:1467] Processing CreateTablet for tablet 7f0d77340b8340b78d05302c7f2d4f87 (DEFAULT_TABLE table=test-table [id=aae363461a284482bd041c8cbb5c0619]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:29:22.527905 28389 tablet_service.cc:1467] Processing CreateTablet for tablet 7f0d77340b8340b78d05302c7f2d4f87 (DEFAULT_TABLE table=test-table [id=aae363461a284482bd041c8cbb5c0619]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:29:22.528765 28603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f0d77340b8340b78d05302c7f2d4f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:29:22.528781 28177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f0d77340b8340b78d05302c7f2d4f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:29:22.528781 28389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7f0d77340b8340b78d05302c7f2d4f87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:29:22.535049 28732 tablet_bootstrap.cc:492] T 7f0d77340b8340b78d05302c7f2d4f87 P e49df4478baa46cbaf98a2b1c1332060: Bootstrap starting.
I20260808 22:29:22.536530 28733 tablet_bootstrap.cc:492] T 7f0d77340b8340b78d05302c7f2d4f87 P 27db2e9af5d04eb2a695be5342e01774: Bootstrap starting.
I20260808 22:29:22.537698 28732 tablet_bootstrap.cc:654] T 7f0d77340b8340b78d05302c7f2d4f87 P e49df4478baa46cbaf98a2b1c1332060: Neither blocks nor log segments found. Creating new log.
I20260808 22:29:22.538431 28732 log.cc:824] T 7f0d77340b8340b78d05302c7f2d4f87 P e49df4478baa46cbaf98a2b1c1332060: Log is configured to *not* fsync() on all Append() calls
I20260808 22:29:22.538630 28733 tablet_bootstrap.cc:654] T 7f0d77340b8340b78d05302c7f2d4f87 P 27db2e9af5d04eb2a695be5342e01774: Neither blocks nor log segments found. Creating new log.
I20260808 22:29:22.539080 28734 tablet_bootstrap.cc:492] T 7f0d77340b8340b78d05302c7f2d4f87 P d84376118b0d44df94afa76d495ae676: Bootstrap starting.
I20260808 22:29:22.539512 28733 log.cc:824] T 7f0d77340b8340b78d05302c7f2d4f87 P 27db2e9af5d04eb2a695be5342e01774: Log is configured to *not* fsync() on all Append() calls
I20260808 22:29:22.540671 28732 tablet_bootstrap.cc:492] T 7f0d77340b8340b78d05302c7f2d4f87 P e49df4478baa46cbaf98a2b1c1332060: No bootstrap required, opened a new log
I20260808 22:29:22.540908 28732 ts_tablet_manager.cc:1397] T 7f0d77340b8340b78d05302c7f2d4f87 P e49df4478baa46cbaf98a2b1c1332060: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 22:29:22.541217 28734 tablet_bootstrap.cc:654] T 7f0d77340b8340b78d05302c7f2d4f87 P d84376118b0d44df94afa76d495ae676: Neither blocks nor log segments found. Creating new log.
I20260808 22:29:22.541523 28733 tablet_bootstrap.cc:492] T 7f0d77340b8340b78d05302c7f2d4f87 P 27db2e9af5d04eb2a695be5342e01774: No bootstrap required, opened a new log
I20260808 22:29:22.541710 28733 ts_tablet_manager.cc:1397] T 7f0d77340b8340b78d05302c7f2d4f87 P 27db2e9af5d04eb2a695be5342e01774: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 22:29:22.541985 28734 log.cc:824] T 7f0d77340b8340b78d05302c7f2d4f87 P d84376118b0d44df94afa76d495ae676: Log is configured to *not* fsync() on all Append() calls
I20260808 22:29:22.543561 28734 tablet_bootstrap.cc:492] T 7f0d77340b8340b78d05302c7f2d4f87 P d84376118b0d44df94afa76d495ae676: No bootstrap required, opened a new log
I20260808 22:29:22.543802 28734 ts_tablet_manager.cc:1397] T 7f0d77340b8340b78d05302c7f2d4f87 P d84376118b0d44df94afa76d495ae676: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260808 22:29:22.544811 28732 raft_consensus.cc:348] T 7f0d77340b8340b78d05302c7f2d4f87 P e49df4478baa46cbaf98a2b1c1332060 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db2e9af5d04eb2a695be5342e01774" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 34481 } } peers { permanent_uuid: "e49df4478baa46cbaf98a2b1c1332060" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36615 } } peers { permanent_uuid: "d84376118b0d44df94afa76d495ae676" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43769 } }
I20260808 22:29:22.545104 28732 raft_consensus.cc:374] T 7f0d77340b8340b78d05302c7f2d4f87 P e49df4478baa46cbaf98a2b1c1332060 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:29:22.545168 28732 raft_consensus.cc:729] T 7f0d77340b8340b78d05302c7f2d4f87 P e49df4478baa46cbaf98a2b1c1332060 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e49df4478baa46cbaf98a2b1c1332060, State: Initialized, Role: FOLLOWER
I20260808 22:29:22.545435 28733 raft_consensus.cc:348] T 7f0d77340b8340b78d05302c7f2d4f87 P 27db2e9af5d04eb2a695be5342e01774 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db2e9af5d04eb2a695be5342e01774" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 34481 } } peers { permanent_uuid: "e49df4478baa46cbaf98a2b1c1332060" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36615 } } peers { permanent_uuid: "d84376118b0d44df94afa76d495ae676" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43769 } }
I20260808 22:29:22.545620 28732 consensus_queue.cc:260] T 7f0d77340b8340b78d05302c7f2d4f87 P e49df4478baa46cbaf98a2b1c1332060 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db2e9af5d04eb2a695be5342e01774" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 34481 } } peers { permanent_uuid: "e49df4478baa46cbaf98a2b1c1332060" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36615 } } peers { permanent_uuid: "d84376118b0d44df94afa76d495ae676" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43769 } }
I20260808 22:29:22.545701 28733 raft_consensus.cc:374] T 7f0d77340b8340b78d05302c7f2d4f87 P 27db2e9af5d04eb2a695be5342e01774 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:29:22.545759 28733 raft_consensus.cc:729] T 7f0d77340b8340b78d05302c7f2d4f87 P 27db2e9af5d04eb2a695be5342e01774 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27db2e9af5d04eb2a695be5342e01774, State: Initialized, Role: FOLLOWER
I20260808 22:29:22.546211 28733 consensus_queue.cc:260] T 7f0d77340b8340b78d05302c7f2d4f87 P 27db2e9af5d04eb2a695be5342e01774 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db2e9af5d04eb2a695be5342e01774" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 34481 } } peers { permanent_uuid: "e49df4478baa46cbaf98a2b1c1332060" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36615 } } peers { permanent_uuid: "d84376118b0d44df94afa76d495ae676" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43769 } }
I20260808 22:29:22.546294 28705 heartbeater.cc:503] Master 127.5.11.254:46669 was elected leader, sending a full tablet report...
I20260808 22:29:22.546654 28732 ts_tablet_manager.cc:1428] T 7f0d77340b8340b78d05302c7f2d4f87 P e49df4478baa46cbaf98a2b1c1332060: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 22:29:22.547149 28733 ts_tablet_manager.cc:1428] T 7f0d77340b8340b78d05302c7f2d4f87 P 27db2e9af5d04eb2a695be5342e01774: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 22:29:22.547467 28495 heartbeater.cc:503] Master 127.5.11.254:46669 was elected leader, sending a full tablet report...
I20260808 22:29:22.547819 28734 raft_consensus.cc:348] T 7f0d77340b8340b78d05302c7f2d4f87 P d84376118b0d44df94afa76d495ae676 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db2e9af5d04eb2a695be5342e01774" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 34481 } } peers { permanent_uuid: "e49df4478baa46cbaf98a2b1c1332060" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36615 } } peers { permanent_uuid: "d84376118b0d44df94afa76d495ae676" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43769 } }
I20260808 22:29:22.548061 28734 raft_consensus.cc:374] T 7f0d77340b8340b78d05302c7f2d4f87 P d84376118b0d44df94afa76d495ae676 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:29:22.548110 28734 raft_consensus.cc:729] T 7f0d77340b8340b78d05302c7f2d4f87 P d84376118b0d44df94afa76d495ae676 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d84376118b0d44df94afa76d495ae676, State: Initialized, Role: FOLLOWER
I20260808 22:29:22.548485 28734 consensus_queue.cc:260] T 7f0d77340b8340b78d05302c7f2d4f87 P d84376118b0d44df94afa76d495ae676 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db2e9af5d04eb2a695be5342e01774" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 34481 } } peers { permanent_uuid: "e49df4478baa46cbaf98a2b1c1332060" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36615 } } peers { permanent_uuid: "d84376118b0d44df94afa76d495ae676" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43769 } }
I20260808 22:29:22.549225 28283 heartbeater.cc:503] Master 127.5.11.254:46669 was elected leader, sending a full tablet report...
I20260808 22:29:22.550415 28734 ts_tablet_manager.cc:1428] T 7f0d77340b8340b78d05302c7f2d4f87 P d84376118b0d44df94afa76d495ae676: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260808 22:29:22.562656 28739 raft_consensus.cc:482] T 7f0d77340b8340b78d05302c7f2d4f87 P 27db2e9af5d04eb2a695be5342e01774 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:29:22.562839 28739 raft_consensus.cc:504] T 7f0d77340b8340b78d05302c7f2d4f87 P 27db2e9af5d04eb2a695be5342e01774 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db2e9af5d04eb2a695be5342e01774" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 34481 } } peers { permanent_uuid: "e49df4478baa46cbaf98a2b1c1332060" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36615 } } peers { permanent_uuid: "d84376118b0d44df94afa76d495ae676" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43769 } }
I20260808 22:29:22.563983 28739 leader_election.cc:288] T 7f0d77340b8340b78d05302c7f2d4f87 P 27db2e9af5d04eb2a695be5342e01774 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e49df4478baa46cbaf98a2b1c1332060 (127.5.11.195:36615), d84376118b0d44df94afa76d495ae676 (127.5.11.193:43769)
I20260808 22:29:22.567889 28497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:22.564273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:36615 (local address 127.5.11.194:46125)
0808 22:29:22.564372 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:22.564389 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:22.564495 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:22.564748 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:22.564750 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:22.564766 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:29:22.564929 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:22.564935 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:22.565781 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:22.565784 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:22.566632 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:22.566642 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:22.567397 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:22.567403 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:22.567522 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:22.567564 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 22:29:22.567635 (+ 71us) client_negotiation.cc:240] Negotiation successful
0808 22:29:22.567718 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24,"spinlock_wait_cycles":23936}
I20260808 22:29:22.567946 28731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:22.564369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:46125 (local address 127.5.11.195:36615)
0808 22:29:22.564482 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:22.564501 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:22.564518 (+ 17us) server_negotiation.cc:407] Connection header received
0808 22:29:22.564584 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:22.564588 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:22.564638 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:22.564732 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:29:22.565064 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:22.565672 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:22.566760 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:22.567274 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:22.567405 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:22.567461 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:22.567764 (+ 303us) server_negotiation.cc:299] Negotiation successful
0808 22:29:22.567833 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 22:29:22.568137 28729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:22.564369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:36881 (local address 127.5.11.193:43769)
0808 22:29:22.564482 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:22.564487 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:22.564813 (+ 326us) server_negotiation.cc:407] Connection header received
0808 22:29:22.564928 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:22.564931 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:22.565000 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:22.565078 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:29:22.565348 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:22.565888 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:22.566946 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:22.567494 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:22.567623 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:22.567672 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:22.567910 (+ 238us) server_negotiation.cc:299] Negotiation successful
0808 22:29:22.568006 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 22:29:22.568508 28741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:22.564273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:43769 (local address 127.5.11.194:36881)
0808 22:29:22.564713 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:22.564728 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:22.564838 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:22.565103 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:22.565105 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:22.565125 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:29:22.565234 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:22.565241 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:22.566009 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:22.566011 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:22.566816 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:22.566827 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:22.567619 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:22.567623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:22.567723 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:22.567775 (+ 52us) client_negotiation.cc:769] Sending connection context
0808 22:29:22.568276 (+ 501us) client_negotiation.cc:240] Negotiation successful
0808 22:29:22.568353 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"mutex_wait_us":96,"thread_start_us":111,"threads_started":1}
I20260808 22:29:22.568578 28640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f0d77340b8340b78d05302c7f2d4f87" candidate_uuid: "27db2e9af5d04eb2a695be5342e01774" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e49df4478baa46cbaf98a2b1c1332060" is_pre_election: true
I20260808 22:29:22.568924 28640 raft_consensus.cc:2454] T 7f0d77340b8340b78d05302c7f2d4f87 P e49df4478baa46cbaf98a2b1c1332060 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27db2e9af5d04eb2a695be5342e01774 in term 0.
I20260808 22:29:22.569176 28212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f0d77340b8340b78d05302c7f2d4f87" candidate_uuid: "27db2e9af5d04eb2a695be5342e01774" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d84376118b0d44df94afa76d495ae676" is_pre_election: true
I20260808 22:29:22.569439 28212 raft_consensus.cc:2454] T 7f0d77340b8340b78d05302c7f2d4f87 P d84376118b0d44df94afa76d495ae676 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 27db2e9af5d04eb2a695be5342e01774 in term 0.
I20260808 22:29:22.569448 28326 leader_election.cc:302] T 7f0d77340b8340b78d05302c7f2d4f87 P 27db2e9af5d04eb2a695be5342e01774 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27db2e9af5d04eb2a695be5342e01774, e49df4478baa46cbaf98a2b1c1332060; no voters:
I20260808 22:29:22.569733 28739 raft_consensus.cc:2781] T 7f0d77340b8340b78d05302c7f2d4f87 P 27db2e9af5d04eb2a695be5342e01774 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:29:22.569837 28739 raft_consensus.cc:482] T 7f0d77340b8340b78d05302c7f2d4f87 P 27db2e9af5d04eb2a695be5342e01774 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:29:22.569895 28739 raft_consensus.cc:3037] T 7f0d77340b8340b78d05302c7f2d4f87 P 27db2e9af5d04eb2a695be5342e01774 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:29:22.570926 28739 raft_consensus.cc:504] T 7f0d77340b8340b78d05302c7f2d4f87 P 27db2e9af5d04eb2a695be5342e01774 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db2e9af5d04eb2a695be5342e01774" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 34481 } } peers { permanent_uuid: "e49df4478baa46cbaf98a2b1c1332060" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36615 } } peers { permanent_uuid: "d84376118b0d44df94afa76d495ae676" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43769 } }
I20260808 22:29:22.571245 28739 leader_election.cc:288] T 7f0d77340b8340b78d05302c7f2d4f87 P 27db2e9af5d04eb2a695be5342e01774 [CANDIDATE]: Term 1 election: Requested vote from peers e49df4478baa46cbaf98a2b1c1332060 (127.5.11.195:36615), d84376118b0d44df94afa76d495ae676 (127.5.11.193:43769)
I20260808 22:29:22.571563 28640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f0d77340b8340b78d05302c7f2d4f87" candidate_uuid: "27db2e9af5d04eb2a695be5342e01774" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e49df4478baa46cbaf98a2b1c1332060"
I20260808 22:29:22.571650 28212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7f0d77340b8340b78d05302c7f2d4f87" candidate_uuid: "27db2e9af5d04eb2a695be5342e01774" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d84376118b0d44df94afa76d495ae676"
I20260808 22:29:22.571693 28640 raft_consensus.cc:3037] T 7f0d77340b8340b78d05302c7f2d4f87 P e49df4478baa46cbaf98a2b1c1332060 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:29:22.571786 28212 raft_consensus.cc:3037] T 7f0d77340b8340b78d05302c7f2d4f87 P d84376118b0d44df94afa76d495ae676 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:29:22.573002 28212 raft_consensus.cc:2454] T 7f0d77340b8340b78d05302c7f2d4f87 P d84376118b0d44df94afa76d495ae676 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27db2e9af5d04eb2a695be5342e01774 in term 1.
I20260808 22:29:22.572993 28640 raft_consensus.cc:2454] T 7f0d77340b8340b78d05302c7f2d4f87 P e49df4478baa46cbaf98a2b1c1332060 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 27db2e9af5d04eb2a695be5342e01774 in term 1.
I20260808 22:29:22.573261 28326 leader_election.cc:302] T 7f0d77340b8340b78d05302c7f2d4f87 P 27db2e9af5d04eb2a695be5342e01774 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27db2e9af5d04eb2a695be5342e01774, e49df4478baa46cbaf98a2b1c1332060; no voters:
I20260808 22:29:22.573451 28739 raft_consensus.cc:2781] T 7f0d77340b8340b78d05302c7f2d4f87 P 27db2e9af5d04eb2a695be5342e01774 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:29:22.573673 28739 raft_consensus.cc:686] T 7f0d77340b8340b78d05302c7f2d4f87 P 27db2e9af5d04eb2a695be5342e01774 [term 1 LEADER]: Becoming Leader. State: Replica: 27db2e9af5d04eb2a695be5342e01774, State: Running, Role: LEADER
I20260808 22:29:22.574026 28739 consensus_queue.cc:237] T 7f0d77340b8340b78d05302c7f2d4f87 P 27db2e9af5d04eb2a695be5342e01774 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db2e9af5d04eb2a695be5342e01774" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 34481 } } peers { permanent_uuid: "e49df4478baa46cbaf98a2b1c1332060" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36615 } } peers { permanent_uuid: "d84376118b0d44df94afa76d495ae676" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43769 } }
I20260808 22:29:22.577070 27980 catalog_manager.cc:5495] T 7f0d77340b8340b78d05302c7f2d4f87 P 27db2e9af5d04eb2a695be5342e01774 reported cstate change: term changed from 0 to 1, leader changed from <none> to 27db2e9af5d04eb2a695be5342e01774 (127.5.11.194). New cstate: current_term: 1 leader_uuid: "27db2e9af5d04eb2a695be5342e01774" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db2e9af5d04eb2a695be5342e01774" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 34481 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e49df4478baa46cbaf98a2b1c1332060" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d84376118b0d44df94afa76d495ae676" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43769 } health_report { overall_health: UNKNOWN } } }
I20260808 22:29:22.588455 28725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:22.585049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45404 (local address 127.5.11.194:34481)
0808 22:29:22.585168 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:22.585171 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:22.585183 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:29:22.585315 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:22.585317 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:22.585365 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:22.585451 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:29:22.585639 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:22.586107 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:22.587011 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:22.587255 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:22.587337 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:22.587593 (+ 256us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:29:22.588117 (+ 524us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:29:22.588208 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:22.588301 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 22:29:22.588357 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 22:29:22.594425 28212 raft_consensus.cc:1261] T 7f0d77340b8340b78d05302c7f2d4f87 P d84376118b0d44df94afa76d495ae676 [term 1 FOLLOWER]: Refusing update from remote peer 27db2e9af5d04eb2a695be5342e01774: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:29:22.594425 28640 raft_consensus.cc:1261] T 7f0d77340b8340b78d05302c7f2d4f87 P e49df4478baa46cbaf98a2b1c1332060 [term 1 FOLLOWER]: Refusing update from remote peer 27db2e9af5d04eb2a695be5342e01774: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:29:22.595273 28743 consensus_queue.cc:1035] T 7f0d77340b8340b78d05302c7f2d4f87 P 27db2e9af5d04eb2a695be5342e01774 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d84376118b0d44df94afa76d495ae676" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43769 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:29:22.595487 28739 consensus_queue.cc:1035] T 7f0d77340b8340b78d05302c7f2d4f87 P 27db2e9af5d04eb2a695be5342e01774 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e49df4478baa46cbaf98a2b1c1332060" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:29:22.608165 28749 mvcc.cc:205] Tried to move back new op lower bound from 7316390553978499072 to 7316390553905676288. Current Snapshot: MvccSnapshot[applied={T|T < 7316390553978499072 or (T in {7316390553978499072})}]
I20260808 22:29:22.611372 27980 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39322:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:29:22.611562 27980 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39322:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:29:22.614156 27980 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662: Sending DeleteTablet for 3 replicas of tablet 7f0d77340b8340b78d05302c7f2d4f87
I20260808 22:29:22.614971 28389 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f0d77340b8340b78d05302c7f2d4f87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:29:22 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:45402
I20260808 22:29:22.614969 28177 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f0d77340b8340b78d05302c7f2d4f87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:29:22 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:47640
I20260808 22:29:22.615489 28765 tablet_replica.cc:330] T 7f0d77340b8340b78d05302c7f2d4f87 P 27db2e9af5d04eb2a695be5342e01774: stopping tablet replica
I20260808 22:29:22.615643 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 28086
I20260808 22:29:22.615833 28765 raft_consensus.cc:2229] T 7f0d77340b8340b78d05302c7f2d4f87 P 27db2e9af5d04eb2a695be5342e01774 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:29:22.615895 28603 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f0d77340b8340b78d05302c7f2d4f87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:29:22 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:51098
I20260808 22:29:22.616192 28765 raft_consensus.cc:2258] T 7f0d77340b8340b78d05302c7f2d4f87 P 27db2e9af5d04eb2a695be5342e01774 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:29:22.620884 28765 ts_tablet_manager.cc:1905] T 7f0d77340b8340b78d05302c7f2d4f87 P 27db2e9af5d04eb2a695be5342e01774: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:29:22.623775 28765 ts_tablet_manager.cc:1918] T 7f0d77340b8340b78d05302c7f2d4f87 P 27db2e9af5d04eb2a695be5342e01774: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:29:22.623876 28765 log.cc:1196] T 7f0d77340b8340b78d05302c7f2d4f87 P 27db2e9af5d04eb2a695be5342e01774: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/wal/wals/7f0d77340b8340b78d05302c7f2d4f87
I20260808 22:29:22.624212 28765 ts_tablet_manager.cc:1939] T 7f0d77340b8340b78d05302c7f2d4f87 P 27db2e9af5d04eb2a695be5342e01774: Deleting consensus metadata
I20260808 22:29:22.625437 27966 catalog_manager.cc:4841] TS 27db2e9af5d04eb2a695be5342e01774 (127.5.11.194:34481): tablet 7f0d77340b8340b78d05302c7f2d4f87 (table test-table [id=aae363461a284482bd041c8cbb5c0619]) successfully deleted
I20260808 22:29:22.627998 28766 tablet_replica.cc:330] T 7f0d77340b8340b78d05302c7f2d4f87 P e49df4478baa46cbaf98a2b1c1332060: stopping tablet replica
I20260808 22:29:22.628330 28766 raft_consensus.cc:2229] T 7f0d77340b8340b78d05302c7f2d4f87 P e49df4478baa46cbaf98a2b1c1332060 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:29:22.628666 28766 raft_consensus.cc:2258] T 7f0d77340b8340b78d05302c7f2d4f87 P e49df4478baa46cbaf98a2b1c1332060 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:29:22.630218 28766 ts_tablet_manager.cc:1905] T 7f0d77340b8340b78d05302c7f2d4f87 P e49df4478baa46cbaf98a2b1c1332060: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:29:22.631407 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 28288
I20260808 22:29:22.637526 28766 ts_tablet_manager.cc:1918] T 7f0d77340b8340b78d05302c7f2d4f87 P e49df4478baa46cbaf98a2b1c1332060: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:29:22.637671 28766 log.cc:1196] T 7f0d77340b8340b78d05302c7f2d4f87 P e49df4478baa46cbaf98a2b1c1332060: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/wal/wals/7f0d77340b8340b78d05302c7f2d4f87
I20260808 22:29:22.638132 28766 ts_tablet_manager.cc:1939] T 7f0d77340b8340b78d05302c7f2d4f87 P e49df4478baa46cbaf98a2b1c1332060: Deleting consensus metadata
W20260808 22:29:22.639520 27964 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.11.193:43769 (error 108)
I20260808 22:29:22.644654 28724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:22.644251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:43769 (local address 127.0.0.1:47646)
0808 22:29:22.644436 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:22.644539 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.193:43769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260808 22:29:22.644865 27964 catalog_manager.cc:4584] TS d84376118b0d44df94afa76d495ae676 (127.5.11.193:43769): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7f0d77340b8340b78d05302c7f2d4f87: Network error: Client connection negotiation failed: client connection to 127.5.11.193:43769: connect: Connection refused (error 111)
I20260808 22:29:22.645217 27966 catalog_manager.cc:4841] TS e49df4478baa46cbaf98a2b1c1332060 (127.5.11.195:36615): tablet 7f0d77340b8340b78d05302c7f2d4f87 (table test-table [id=aae363461a284482bd041c8cbb5c0619]) successfully deleted
I20260808 22:29:22.650451 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 28505
I20260808 22:29:22.660311 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 27938
I20260808 22:29:22.672469 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:46669
--webserver_interface=127.5.11.254
--webserver_port=36703
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:34707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:46669
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:29:22.785044 28769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:29:22.785285 28769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:29:22.785324 28769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:29:22.789304 28769 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:29:22.789363 28769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:29:22.789382 28769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:29:22.789399 28769 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:29:22.789414 28769 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:29:22.794543 28769 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:34707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:46669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:46669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=36703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:29:22.795571 28769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:29:22.796612 28769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:29:22.811450 28776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:22.811450 28778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:22.811532 28783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:29:22.811775 28769 server_base.cc:1034] running on GCE node
I20260808 22:29:22.812464 28769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:29:22.813542 28769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:29:22.814715 28769 hybrid_clock.cc:648] HybridClock initialized: now 1786228162814682 us; error 48 us; skew 500 ppm
Aug 08 22:29:22 dist-test-slave-7f01 krb5kdc[27921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228162, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:29:22.817526 28769 init.cc:377] Logged in from keytab as kudu/127.5.11.254@KRBTEST.COM (short username kudu)
I20260808 22:29:22.818367 28769 webserver.cc:460] Webserver started at http://127.5.11.254:36703/ using document root <none> and password file <none>
I20260808 22:29:22.818970 28769 fs_manager.cc:359] Metadata directory not provided
I20260808 22:29:22.819075 28769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:29:22.823017 28769 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:29:22.825182 28790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:22.826375 28769 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:29:22.826526 28769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "ff42823b2ab54e03bac154722af86662"
format_stamp: "Formatted at 2026-08-08 22:29:21 on dist-test-slave-7f01"
server_key: "4fe00a98e8959ff4aae4e018e4c16ce3"
server_key_iv: "bbe40d0c30c37a9cd9f824e048f96b72"
server_key_version: "encryptionkey@0"
I20260808 22:29:22.827114 28769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:29:22.849046 28769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:29:22.852103 28769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:29:22.852314 28769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:29:22.859916 28769 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:46669
I20260808 22:29:22.860224 28865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:46669 every 8 connection(s)
I20260808 22:29:22.861073 28769 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:29:22.865968 28867 sys_catalog.cc:263] Verifying existing consensus state
I20260808 22:29:22.867565 28867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662: Bootstrap starting.
I20260808 22:29:22.869398 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 28769
I20260808 22:29:22.869920 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.193:43769
--local_ip_for_outbound_sockets=127.5.11.193
--tserver_master_addrs=127.5.11.254:46669
--webserver_port=40453
--webserver_interface=127.5.11.193
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:34707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:29:22.885399 28867 log.cc:824] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662: Log is configured to *not* fsync() on all Append() calls
I20260808 22:29:22.894191 28867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 22:29:22.894667 28867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662: Bootstrap complete.
I20260808 22:29:22.898909 28867 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff42823b2ab54e03bac154722af86662" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 46669 } }
I20260808 22:29:22.899293 28867 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff42823b2ab54e03bac154722af86662, State: Initialized, Role: FOLLOWER
I20260808 22:29:22.899849 28867 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff42823b2ab54e03bac154722af86662" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 46669 } }
I20260808 22:29:22.899969 28867 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:29:22.900058 28867 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:29:22.900170 28867 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [term 1 FOLLOWER]: Advancing to term 2
I20260808 22:29:22.902030 28867 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff42823b2ab54e03bac154722af86662" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 46669 } }
I20260808 22:29:22.902372 28867 leader_election.cc:302] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff42823b2ab54e03bac154722af86662; no voters:
I20260808 22:29:22.902719 28867 leader_election.cc:288] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 22:29:22.902801 28871 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 22:29:22.903025 28871 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [term 2 LEADER]: Becoming Leader. State: Replica: ff42823b2ab54e03bac154722af86662, State: Running, Role: LEADER
I20260808 22:29:22.903323 28871 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff42823b2ab54e03bac154722af86662" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 46669 } }
I20260808 22:29:22.904029 28867 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:29:22.904665 28872 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ff42823b2ab54e03bac154722af86662" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff42823b2ab54e03bac154722af86662" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 46669 } } }
I20260808 22:29:22.904793 28872 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [sys.catalog]: This master's current role is: LEADER
I20260808 22:29:22.905449 28875 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff42823b2ab54e03bac154722af86662. Latest consensus state: current_term: 2 leader_uuid: "ff42823b2ab54e03bac154722af86662" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff42823b2ab54e03bac154722af86662" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 46669 } } }
I20260808 22:29:22.905619 28875 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [sys.catalog]: This master's current role is: LEADER
I20260808 22:29:22.905931 28884 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:29:22.910868 28884 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:29:22.911479 28884 catalog_manager.cc:1259] Loaded cluster ID: 1bffbea253484f77945a73d2ea733db3
I20260808 22:29:22.911553 28884 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:29:22.913007 28884 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260808 22:29:22.917654 28884 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260808 22:29:22.922950 28884 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:29:22.923660 28884 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662: Loaded TSK: 0
I20260808 22:29:22.924896 28884 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 22:29:22.990654 28869 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:29:22.990911 28869 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:29:22.990953 28869 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:29:22.995201 28869 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:29:22.995262 28869 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:29:22.995321 28869 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.193
I20260808 22:29:23.000582 28869 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:34707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.193:43769
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.11.193
--webserver_port=40453
--tserver_master_addrs=127.5.11.254:46669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28869
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.193
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:29:23.001588 28869 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:29:23.002612 28869 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:29:23.012943 28908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:23.012945 28907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:23.012943 28910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:29:23.013612 28869 server_base.cc:1034] running on GCE node
I20260808 22:29:23.013954 28869 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:29:23.014583 28869 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:29:23.015805 28869 hybrid_clock.cc:648] HybridClock initialized: now 1786228163015767 us; error 51 us; skew 500 ppm
Aug 08 22:29:23 dist-test-slave-7f01 krb5kdc[27921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228163, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:29:23.018846 28869 init.cc:377] Logged in from keytab as kudu/127.5.11.193@KRBTEST.COM (short username kudu)
I20260808 22:29:23.019835 28869 webserver.cc:460] Webserver started at http://127.5.11.193:40453/ using document root <none> and password file <none>
I20260808 22:29:23.020406 28869 fs_manager.cc:359] Metadata directory not provided
I20260808 22:29:23.020483 28869 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:29:23.024348 28869 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:29:23.026513 28921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:23.028088 28869 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:29:23.028213 28869 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/wal
uuid: "d84376118b0d44df94afa76d495ae676"
format_stamp: "Formatted at 2026-08-08 22:29:21 on dist-test-slave-7f01"
server_key: "93dc6161b588eb8ca7158c229d8fa5a9"
server_key_iv: "e90ee9ab882aac8153fdc2e267b83b5b"
server_key_version: "encryptionkey@0"
I20260808 22:29:23.028861 28869 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:29:23.055632 28869 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:29:23.059402 28869 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:29:23.059614 28869 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:29:23.060351 28869 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:29:23.061388 28931 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 22:29:23.065672 28869 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 22:29:23.065769 28869 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260808 22:29:23.065884 28869 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 22:29:23.067641 28869 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 22:29:23.067754 28869 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260808 22:29:23.068065 28931 tablet_bootstrap.cc:492] T 7f0d77340b8340b78d05302c7f2d4f87 P d84376118b0d44df94afa76d495ae676: Bootstrap starting.
I20260808 22:29:23.078697 28869 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.193:43769
I20260808 22:29:23.078769 29098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.193:43769 every 8 connection(s)
I20260808 22:29:23.079675 28869 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
I20260808 22:29:23.086531 28931 log.cc:824] T 7f0d77340b8340b78d05302c7f2d4f87 P d84376118b0d44df94afa76d495ae676: Log is configured to *not* fsync() on all Append() calls
I20260808 22:29:23.088267 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 28869
I20260808 22:29:23.088806 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.194:34481
--local_ip_for_outbound_sockets=127.5.11.194
--tserver_master_addrs=127.5.11.254:46669
--webserver_port=37437
--webserver_interface=127.5.11.194
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:34707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 22:29:23 dist-test-slave-7f01 krb5kdc[27921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228163, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:29:23.094470 28931 tablet_bootstrap.cc:492] T 7f0d77340b8340b78d05302c7f2d4f87 P d84376118b0d44df94afa76d495ae676: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 22:29:23.094941 28931 tablet_bootstrap.cc:492] T 7f0d77340b8340b78d05302c7f2d4f87 P d84376118b0d44df94afa76d495ae676: Bootstrap complete.
I20260808 22:29:23.096398 28931 ts_tablet_manager.cc:1397] T 7f0d77340b8340b78d05302c7f2d4f87 P d84376118b0d44df94afa76d495ae676: Time spent bootstrapping tablet: real 0.029s user 0.024s sys 0.003s
I20260808 22:29:23.098062 29105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:23.081868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.193:40751)
0808 22:29:23.082447 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:23.082488 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:23.083284 (+ 796us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:23.084486 (+ 1202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:23.084495 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:23.084926 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:29:23.085397 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:23.085412 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:23.086746 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:23.086750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:23.087978 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:23.087988 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:23.089167 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:23.089174 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:23.089370 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:23.090093 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:29:23.090113 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:29:23.092117 (+ 2004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:29:23.094379 (+ 2262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:23.094385 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:23.094397 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:23.094775 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:23.095226 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:23.095232 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:23.095237 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:23.095419 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:23.096235 (+ 816us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:29:23.096244 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:29:23.096553 (+ 309us) client_negotiation.cc:769] Sending connection context
0808 22:29:23.096801 (+ 248us) client_negotiation.cc:240] Negotiation successful
0808 22:29:23.097126 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":188,"threads_started":1}
I20260808 22:29:23.098693 29104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:23.081792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:40751 (local address 127.5.11.254:46669)
0808 22:29:23.082171 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:23.082209 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:23.082620 (+ 411us) server_negotiation.cc:407] Connection header received
0808 22:29:23.083719 (+ 1099us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:23.083729 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:23.084097 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:23.084424 (+ 327us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:23.085581 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:23.086557 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:23.088140 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:23.088908 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:23.089101 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:23.092374 (+ 3273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:23.092384 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:23.092404 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:23.092448 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:23.094239 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:23.094930 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:23.094938 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:23.094947 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:23.095119 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:23.095578 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:23.095584 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:23.095589 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:23.096112 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:23.096274 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:23.096977 (+ 703us) server_negotiation.cc:299] Negotiation successful
0808 22:29:23.097328 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":129,"threads_started":1}
I20260808 22:29:23.099243 28931 raft_consensus.cc:348] T 7f0d77340b8340b78d05302c7f2d4f87 P d84376118b0d44df94afa76d495ae676 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db2e9af5d04eb2a695be5342e01774" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 34481 } } peers { permanent_uuid: "e49df4478baa46cbaf98a2b1c1332060" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36615 } } peers { permanent_uuid: "d84376118b0d44df94afa76d495ae676" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43769 } }
I20260808 22:29:23.099790 28931 raft_consensus.cc:729] T 7f0d77340b8340b78d05302c7f2d4f87 P d84376118b0d44df94afa76d495ae676 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d84376118b0d44df94afa76d495ae676, State: Initialized, Role: FOLLOWER
I20260808 22:29:23.101245 28931 consensus_queue.cc:260] T 7f0d77340b8340b78d05302c7f2d4f87 P d84376118b0d44df94afa76d495ae676 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "27db2e9af5d04eb2a695be5342e01774" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 34481 } } peers { permanent_uuid: "e49df4478baa46cbaf98a2b1c1332060" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36615 } } peers { permanent_uuid: "d84376118b0d44df94afa76d495ae676" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43769 } }
I20260808 22:29:23.101611 29101 heartbeater.cc:347] Connected to a master server at 127.5.11.254:46669
I20260808 22:29:23.101883 29101 heartbeater.cc:464] Registering TS with master...
I20260808 22:29:23.102279 28931 ts_tablet_manager.cc:1428] T 7f0d77340b8340b78d05302c7f2d4f87 P d84376118b0d44df94afa76d495ae676: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260808 22:29:23.102560 29101 heartbeater.cc:511] Master 127.5.11.254:46669 requested a full tablet report, sending...
I20260808 22:29:23.105692 28817 ts_manager.cc:194] Registered new tserver with Master: d84376118b0d44df94afa76d495ae676 (127.5.11.193:43769)
I20260808 22:29:23.109264 28817 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.193@KRBTEST.COM'} at 127.5.11.193:40751
I20260808 22:29:23.112071 29101 heartbeater.cc:503] Master 127.5.11.254:46669 was elected leader, sending a full tablet report...
Aug 08 22:29:23 dist-test-slave-7f01 krb5kdc[27921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228162, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for kudu/127.5.11.193@KRBTEST.COM
I20260808 22:29:23.119097 29112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:23.108683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:43769 (local address 127.0.0.1:47648)
0808 22:29:23.109372 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:23.109419 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:23.109556 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:23.110076 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:23.110083 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:23.110499 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:29:23.110611 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:23.110618 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:23.111575 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:23.111580 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:23.112279 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:23.112289 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:23.113026 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:23.113030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:23.113153 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:23.113791 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:29:23.113806 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:29:23.115334 (+ 1528us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:29:23.117477 (+ 2143us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:23.117485 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:23.117490 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:23.117717 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:23.118039 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:23.118041 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:23.118043 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:23.118123 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:23.118624 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:29:23.118629 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:29:23.118721 (+ 92us) client_negotiation.cc:769] Sending connection context
0808 22:29:23.118830 (+ 109us) client_negotiation.cc:240] Negotiation successful
0808 22:29:23.118974 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":108,"threads_started":1}
I20260808 22:29:23.119253 29113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:23.108769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47648 (local address 127.5.11.193:43769)
0808 22:29:23.109106 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:23.109117 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:23.109517 (+ 400us) server_negotiation.cc:407] Connection header received
0808 22:29:23.109714 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:23.109719 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:23.109921 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:23.110052 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:23.110739 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:23.111452 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:23.112409 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:23.112904 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:23.113019 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:23.115482 (+ 2463us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:23.115492 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:23.115507 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:23.115536 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:23.117373 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:23.117831 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:23.117837 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:23.117839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:23.117942 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:23.118217 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:23.118220 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:23.118222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:23.118517 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:23.118656 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:23.118941 (+ 285us) server_negotiation.cc:299] Negotiation successful
0808 22:29:23.119103 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":85,"threads_started":1}
I20260808 22:29:23.120204 28993 tablet_service.cc:1514] Processing DeleteTablet for tablet 7f0d77340b8340b78d05302c7f2d4f87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:29:22 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:47648
I20260808 22:29:23.121032 29116 tablet_replica.cc:330] stopping tablet replica
I20260808 22:29:23.121330 29116 raft_consensus.cc:2229] T 7f0d77340b8340b78d05302c7f2d4f87 P d84376118b0d44df94afa76d495ae676 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:29:23.121585 29116 raft_consensus.cc:2258] T 7f0d77340b8340b78d05302c7f2d4f87 P d84376118b0d44df94afa76d495ae676 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:29:23.122535 29116 ts_tablet_manager.cc:1905] T 7f0d77340b8340b78d05302c7f2d4f87 P d84376118b0d44df94afa76d495ae676: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:29:23.126076 29116 ts_tablet_manager.cc:1918] T 7f0d77340b8340b78d05302c7f2d4f87 P d84376118b0d44df94afa76d495ae676: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:29:23.126160 29116 log.cc:1196] T 7f0d77340b8340b78d05302c7f2d4f87 P d84376118b0d44df94afa76d495ae676: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/wal/wals/7f0d77340b8340b78d05302c7f2d4f87
I20260808 22:29:23.126485 29116 ts_tablet_manager.cc:1939] T 7f0d77340b8340b78d05302c7f2d4f87 P d84376118b0d44df94afa76d495ae676: Deleting consensus metadata
I20260808 22:29:23.128139 28794 catalog_manager.cc:4841] TS d84376118b0d44df94afa76d495ae676 (127.5.11.193:43769): tablet 7f0d77340b8340b78d05302c7f2d4f87 (table test-table [id=aae363461a284482bd041c8cbb5c0619]) successfully deleted
W20260808 22:29:23.212229 29110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:29:23.212504 29110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:29:23.212589 29110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:29:23.216470 29110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:29:23.216547 29110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:29:23.216630 29110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.194
I20260808 22:29:23.221698 29110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:34707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.194:34481
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.11.194
--webserver_port=37437
--tserver_master_addrs=127.5.11.254:46669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.194
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:29:23.222744 29110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:29:23.223886 29110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:29:23.233733 29123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:23.233736 29124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:23.233783 29126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:29:23.234004 29110 server_base.cc:1034] running on GCE node
I20260808 22:29:23.234532 29110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:29:23.235177 29110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:29:23.236367 29110 hybrid_clock.cc:648] HybridClock initialized: now 1786228163236351 us; error 26 us; skew 500 ppm
Aug 08 22:29:23 dist-test-slave-7f01 krb5kdc[27921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228163, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:29:23.239054 29110 init.cc:377] Logged in from keytab as kudu/127.5.11.194@KRBTEST.COM (short username kudu)
I20260808 22:29:23.240010 29110 webserver.cc:460] Webserver started at http://127.5.11.194:37437/ using document root <none> and password file <none>
I20260808 22:29:23.240577 29110 fs_manager.cc:359] Metadata directory not provided
I20260808 22:29:23.240691 29110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:29:23.244537 29110 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:29:23.246547 29134 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:23.247843 29110 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:29:23.247988 29110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/wal
uuid: "27db2e9af5d04eb2a695be5342e01774"
format_stamp: "Formatted at 2026-08-08 22:29:22 on dist-test-slave-7f01"
server_key: "52dedd1382beac0ef83ce7a07dfe95bf"
server_key_iv: "eed92f487c2011f7c09b785ca8f5538b"
server_key_version: "encryptionkey@0"
I20260808 22:29:23.248545 29110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:29:23.261188 29110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:29:23.263908 29110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:29:23.264099 29110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:29:23.264674 29110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:29:23.265509 29110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:29:23.265592 29110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:23.265682 29110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:29:23.265733 29110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:23.274873 29110 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.194:34481
I20260808 22:29:23.274895 29303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.194:34481 every 8 connection(s)
I20260808 22:29:23.275938 29110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
I20260808 22:29:23.279680 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 29110
I20260808 22:29:23.280251 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.195:36615
--local_ip_for_outbound_sockets=127.5.11.195
--tserver_master_addrs=127.5.11.254:46669
--webserver_port=43573
--webserver_interface=127.5.11.195
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:34707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 22:29:23 dist-test-slave-7f01 krb5kdc[27921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228163, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:29:23.291049 29104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:23.277717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:60247 (local address 127.5.11.254:46669)
0808 22:29:23.277873 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:23.277878 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:23.278954 (+ 1076us) server_negotiation.cc:407] Connection header received
0808 22:29:23.280005 (+ 1051us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:23.280007 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:23.280055 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:23.280127 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:23.281723 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:23.282408 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:23.283526 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:23.283940 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:23.284027 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:23.287072 (+ 3045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:23.287075 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:23.287077 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:23.287101 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:23.288706 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:23.289366 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:23.289369 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:23.289370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:23.289458 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:23.289873 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:23.289875 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:23.289876 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:23.290045 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:23.290119 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:23.290812 (+ 693us) server_negotiation.cc:299] Negotiation successful
0808 22:29:23.290922 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 22:29:23.292383 29310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:23.278150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.194:60247)
0808 22:29:23.278813 (+ 663us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:23.278853 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:23.279777 (+ 924us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:23.280314 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:23.280324 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:23.280873 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:29:23.281566 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:23.281589 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:23.282550 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:23.282552 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:23.283400 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:23.283410 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:23.284098 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:23.284105 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:23.284338 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:23.284964 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:29:23.284988 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:29:23.286894 (+ 1906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:29:23.288840 (+ 1946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:23.288849 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:23.288867 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:23.289262 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:23.289555 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:23.289561 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:23.289564 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:23.289769 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:23.290157 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:29:23.290166 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:29:23.290525 (+ 359us) client_negotiation.cc:769] Sending connection context
0808 22:29:23.290769 (+ 244us) client_negotiation.cc:240] Negotiation successful
0808 22:29:23.291153 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":111,"threads_started":1}
I20260808 22:29:23.293615 29307 heartbeater.cc:347] Connected to a master server at 127.5.11.254:46669
I20260808 22:29:23.293889 29307 heartbeater.cc:464] Registering TS with master...
I20260808 22:29:23.294485 29307 heartbeater.cc:511] Master 127.5.11.254:46669 requested a full tablet report, sending...
I20260808 22:29:23.295709 28817 ts_manager.cc:194] Registered new tserver with Master: 27db2e9af5d04eb2a695be5342e01774 (127.5.11.194:34481)
I20260808 22:29:23.296394 28817 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.194@KRBTEST.COM'} at 127.5.11.194:60247
W20260808 22:29:23.397568 29312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:29:23.397851 29312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:29:23.397927 29312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:29:23.401821 29312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:29:23.401909 29312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:29:23.402009 29312 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.195
I20260808 22:29:23.407438 29312 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:34707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.195:36615
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.11.195
--webserver_port=43573
--tserver_master_addrs=127.5.11.254:46669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.195
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:29:23.408591 29312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:29:23.409679 29312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:29:23.419137 29325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:29:23.419199 29312 server_base.cc:1034] running on GCE node
W20260808 22:29:23.419142 29319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:23.419359 29320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:29:23.419782 29312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:29:23.420444 29312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:29:23.421634 29312 hybrid_clock.cc:648] HybridClock initialized: now 1786228163421592 us; error 53 us; skew 500 ppm
Aug 08 22:29:23 dist-test-slave-7f01 krb5kdc[27921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228163, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:29:23.424333 29312 init.cc:377] Logged in from keytab as kudu/127.5.11.195@KRBTEST.COM (short username kudu)
I20260808 22:29:23.425148 29312 webserver.cc:460] Webserver started at http://127.5.11.195:43573/ using document root <none> and password file <none>
I20260808 22:29:23.425787 29312 fs_manager.cc:359] Metadata directory not provided
I20260808 22:29:23.425899 29312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:29:23.429742 29312 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:29:23.431921 29334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:23.433116 29312 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 22:29:23.433264 29312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/wal
uuid: "e49df4478baa46cbaf98a2b1c1332060"
format_stamp: "Formatted at 2026-08-08 22:29:22 on dist-test-slave-7f01"
server_key: "b5d4f5261b0ab3ff58fe0ef707124d37"
server_key_iv: "aeb17ba00f9f4fd3223a8e3da3bfe8bc"
server_key_version: "encryptionkey@0"
I20260808 22:29:23.433844 29312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:29:23.449440 29312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:29:23.452687 29312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:29:23.452880 29312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:29:23.453451 29312 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:29:23.454264 29312 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:29:23.454342 29312 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:23.454414 29312 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:29:23.454460 29312 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:23.463708 29312 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.195:36615
I20260808 22:29:23.463774 29519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.195:36615 every 8 connection(s)
I20260808 22:29:23.464793 29312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
I20260808 22:29:23.468318 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 29312
Aug 08 22:29:23 dist-test-slave-7f01 krb5kdc[27921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228163, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:29:23.478364 29526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:23.469010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39328 (local address 127.5.11.254:46669)
0808 22:29:23.469602 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:23.469605 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:23.469615 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:29:23.469646 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:23.469648 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:23.469680 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:23.469750 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:23.470671 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:23.471354 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:23.472558 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:23.472840 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:23.472933 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:23.474676 (+ 1743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:23.474680 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:23.474683 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:23.474728 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:23.476621 (+ 1893us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:23.477073 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:23.477077 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:23.477078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:23.477171 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:23.477466 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:23.477470 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:23.477471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:23.477684 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:23.477904 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:23.478057 (+ 153us) server_negotiation.cc:299] Negotiation successful
0808 22:29:23.478186 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":116,"threads_started":1}
I20260808 22:29:23.479831 29104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:23.466725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:33537 (local address 127.5.11.254:46669)
0808 22:29:23.466884 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:23.466888 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:23.467835 (+ 947us) server_negotiation.cc:407] Connection header received
0808 22:29:23.468802 (+ 967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:23.468806 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:23.468870 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:23.468984 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:23.470417 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:23.471186 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:23.472775 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:23.473088 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:23.473162 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:23.476102 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:23.476109 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:23.476113 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:23.476156 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:23.477585 (+ 1429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:23.478230 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:23.478233 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:23.478238 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:23.478352 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:23.478736 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:23.478744 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:23.478745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:23.478953 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:23.479049 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:23.479578 (+ 529us) server_negotiation.cc:299] Negotiation successful
0808 22:29:23.479669 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 22:29:23.480764 29522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:23.467046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.195:33537)
0808 22:29:23.467687 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:23.467727 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:23.468582 (+ 855us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:23.469189 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:23.469198 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:23.469574 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:29:23.470242 (+ 668us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:23.470266 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:23.471311 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:23.471314 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:23.472667 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:23.472677 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:23.473250 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:23.473254 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:23.473431 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:23.473931 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:29:23.473954 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:29:23.475939 (+ 1985us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:29:23.477698 (+ 1759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:23.477706 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:23.477721 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:23.478098 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:23.478457 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:23.478461 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:23.478463 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:23.478633 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:23.479068 (+ 435us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:29:23.479074 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:29:23.479359 (+ 285us) client_negotiation.cc:769] Sending connection context
0808 22:29:23.479541 (+ 182us) client_negotiation.cc:240] Negotiation successful
0808 22:29:23.479886 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":103,"threads_started":1}
I20260808 22:29:23.481801 29520 heartbeater.cc:347] Connected to a master server at 127.5.11.254:46669
I20260808 22:29:23.482040 29520 heartbeater.cc:464] Registering TS with master...
I20260808 22:29:23.482542 29520 heartbeater.cc:511] Master 127.5.11.254:46669 requested a full tablet report, sending...
I20260808 22:29:23.483522 28817 ts_manager.cc:194] Registered new tserver with Master: e49df4478baa46cbaf98a2b1c1332060 (127.5.11.195:36615)
I20260808 22:29:23.484127 28817 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.195@KRBTEST.COM'} at 127.5.11.195:33537
I20260808 22:29:23.492266 5167 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:29:23.497588 29104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:23.492996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39330 (local address 127.5.11.254:46669)
0808 22:29:23.493141 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:23.493145 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:23.493750 (+ 605us) server_negotiation.cc:407] Connection header received
0808 22:29:23.493933 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:23.493936 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:23.494072 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:23.494203 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:29:23.494479 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:23.495294 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:23.496308 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:23.496519 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:23.496598 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:23.496872 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:29:23.497236 (+ 364us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:29:23.497323 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:23.497381 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 22:29:23.497424 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 22:29:23.498201 28817 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:39330:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:29:23.501788 28817 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:29:23.509788 28993 tablet_service.cc:1467] Processing CreateTablet for tablet 8c018d6838314fc882dbe6c0d1bdb10a (DEFAULT_TABLE table=test-table [id=10147643ecad462db6fd371108ab6986]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:29:23.510488 28993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c018d6838314fc882dbe6c0d1bdb10a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:29:23.513073 28931 tablet_bootstrap.cc:492] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676: Bootstrap starting.
I20260808 22:29:23.513882 28931 tablet_bootstrap.cc:654] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676: Neither blocks nor log segments found. Creating new log.
I20260808 22:29:23.515033 29535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:23.509474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51100 (local address 127.5.11.195:36615)
0808 22:29:23.509800 (+ 326us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:23.509811 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:23.509831 (+ 20us) server_negotiation.cc:407] Connection header received
0808 22:29:23.509896 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:23.509903 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:23.510120 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:23.510292 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:29:23.510775 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:23.511908 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:23.513533 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:23.514272 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:23.514390 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:23.514561 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:23.514740 (+ 179us) server_negotiation.cc:299] Negotiation successful
0808 22:29:23.514853 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":119,"threads_started":1}
I20260808 22:29:23.514999 29112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:23.508777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:36615 (local address 127.0.0.1:51100)
0808 22:29:23.508877 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:23.508892 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:23.508999 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:23.510560 (+ 1561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:23.510562 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:23.510579 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:29:23.510652 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:23.510657 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:23.512193 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:23.512196 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:23.513286 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:23.513296 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:23.514414 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:23.514419 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:23.514554 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:23.514610 (+ 56us) client_negotiation.cc:769] Sending connection context
0808 22:29:23.514704 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 22:29:23.514794 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":3}
I20260808 22:29:23.515538 28931 tablet_bootstrap.cc:492] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676: No bootstrap required, opened a new log
I20260808 22:29:23.515627 29531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:23.509039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:34481 (local address 127.0.0.1:45410)
0808 22:29:23.509280 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:23.509295 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:23.509381 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:23.510287 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:23.510290 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:23.510309 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:29:23.510423 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:23.510429 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:23.511932 (+ 1503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:23.511937 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:23.513237 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:23.513249 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:23.514416 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:23.514420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:23.515290 (+ 870us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:23.515318 (+ 28us) client_negotiation.cc:769] Sending connection context
0808 22:29:23.515412 (+ 94us) client_negotiation.cc:240] Negotiation successful
0808 22:29:23.515488 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"mutex_wait_us":751,"thread_start_us":112,"threads_started":1}
I20260808 22:29:23.515637 28931 ts_tablet_manager.cc:1397] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:29:23.515851 29532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:23.509279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45410 (local address 127.5.11.194:34481)
0808 22:29:23.509771 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:23.509783 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:23.509809 (+ 26us) server_negotiation.cc:407] Connection header received
0808 22:29:23.509897 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:23.509903 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:23.510120 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:23.510292 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:29:23.510548 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:23.511777 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:23.513406 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:23.514265 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:23.514466 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:23.514585 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:23.515469 (+ 884us) server_negotiation.cc:299] Negotiation successful
0808 22:29:23.515567 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":159,"threads_started":1}
I20260808 22:29:23.516283 28931 raft_consensus.cc:348] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e49df4478baa46cbaf98a2b1c1332060" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36615 } } peers { permanent_uuid: "27db2e9af5d04eb2a695be5342e01774" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 34481 } } peers { permanent_uuid: "d84376118b0d44df94afa76d495ae676" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43769 } }
I20260808 22:29:23.516426 28931 raft_consensus.cc:374] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:29:23.516469 28931 raft_consensus.cc:729] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d84376118b0d44df94afa76d495ae676, State: Initialized, Role: FOLLOWER
I20260808 22:29:23.518412 29412 tablet_service.cc:1467] Processing CreateTablet for tablet 8c018d6838314fc882dbe6c0d1bdb10a (DEFAULT_TABLE table=test-table [id=10147643ecad462db6fd371108ab6986]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:29:23.518399 29199 tablet_service.cc:1467] Processing CreateTablet for tablet 8c018d6838314fc882dbe6c0d1bdb10a (DEFAULT_TABLE table=test-table [id=10147643ecad462db6fd371108ab6986]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:29:23.519328 29199 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c018d6838314fc882dbe6c0d1bdb10a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:29:23.519345 29412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8c018d6838314fc882dbe6c0d1bdb10a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:29:23.516640 28931 consensus_queue.cc:260] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e49df4478baa46cbaf98a2b1c1332060" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36615 } } peers { permanent_uuid: "27db2e9af5d04eb2a695be5342e01774" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 34481 } } peers { permanent_uuid: "d84376118b0d44df94afa76d495ae676" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43769 } }
I20260808 22:29:23.524449 28931 ts_tablet_manager.cc:1428] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676: Time spent starting tablet: real 0.009s user 0.000s sys 0.002s
I20260808 22:29:23.526393 29537 tablet_bootstrap.cc:492] T 8c018d6838314fc882dbe6c0d1bdb10a P 27db2e9af5d04eb2a695be5342e01774: Bootstrap starting.
I20260808 22:29:23.526885 29538 tablet_bootstrap.cc:492] T 8c018d6838314fc882dbe6c0d1bdb10a P e49df4478baa46cbaf98a2b1c1332060: Bootstrap starting.
I20260808 22:29:23.529138 29538 tablet_bootstrap.cc:654] T 8c018d6838314fc882dbe6c0d1bdb10a P e49df4478baa46cbaf98a2b1c1332060: Neither blocks nor log segments found. Creating new log.
I20260808 22:29:23.529531 29537 tablet_bootstrap.cc:654] T 8c018d6838314fc882dbe6c0d1bdb10a P 27db2e9af5d04eb2a695be5342e01774: Neither blocks nor log segments found. Creating new log.
I20260808 22:29:23.530270 29538 log.cc:824] T 8c018d6838314fc882dbe6c0d1bdb10a P e49df4478baa46cbaf98a2b1c1332060: Log is configured to *not* fsync() on all Append() calls
I20260808 22:29:23.530261 29537 log.cc:824] T 8c018d6838314fc882dbe6c0d1bdb10a P 27db2e9af5d04eb2a695be5342e01774: Log is configured to *not* fsync() on all Append() calls
I20260808 22:29:23.532284 29537 tablet_bootstrap.cc:492] T 8c018d6838314fc882dbe6c0d1bdb10a P 27db2e9af5d04eb2a695be5342e01774: No bootstrap required, opened a new log
I20260808 22:29:23.532294 29538 tablet_bootstrap.cc:492] T 8c018d6838314fc882dbe6c0d1bdb10a P e49df4478baa46cbaf98a2b1c1332060: No bootstrap required, opened a new log
I20260808 22:29:23.532491 29537 ts_tablet_manager.cc:1397] T 8c018d6838314fc882dbe6c0d1bdb10a P 27db2e9af5d04eb2a695be5342e01774: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 22:29:23.532492 29538 ts_tablet_manager.cc:1397] T 8c018d6838314fc882dbe6c0d1bdb10a P e49df4478baa46cbaf98a2b1c1332060: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 22:29:23.535391 29537 raft_consensus.cc:348] T 8c018d6838314fc882dbe6c0d1bdb10a P 27db2e9af5d04eb2a695be5342e01774 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e49df4478baa46cbaf98a2b1c1332060" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36615 } } peers { permanent_uuid: "27db2e9af5d04eb2a695be5342e01774" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 34481 } } peers { permanent_uuid: "d84376118b0d44df94afa76d495ae676" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43769 } }
I20260808 22:29:23.535648 29537 raft_consensus.cc:374] T 8c018d6838314fc882dbe6c0d1bdb10a P 27db2e9af5d04eb2a695be5342e01774 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:29:23.535532 29538 raft_consensus.cc:348] T 8c018d6838314fc882dbe6c0d1bdb10a P e49df4478baa46cbaf98a2b1c1332060 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e49df4478baa46cbaf98a2b1c1332060" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36615 } } peers { permanent_uuid: "27db2e9af5d04eb2a695be5342e01774" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 34481 } } peers { permanent_uuid: "d84376118b0d44df94afa76d495ae676" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43769 } }
I20260808 22:29:23.535727 29537 raft_consensus.cc:729] T 8c018d6838314fc882dbe6c0d1bdb10a P 27db2e9af5d04eb2a695be5342e01774 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27db2e9af5d04eb2a695be5342e01774, State: Initialized, Role: FOLLOWER
I20260808 22:29:23.535804 29538 raft_consensus.cc:374] T 8c018d6838314fc882dbe6c0d1bdb10a P e49df4478baa46cbaf98a2b1c1332060 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:29:23.535877 29538 raft_consensus.cc:729] T 8c018d6838314fc882dbe6c0d1bdb10a P e49df4478baa46cbaf98a2b1c1332060 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e49df4478baa46cbaf98a2b1c1332060, State: Initialized, Role: FOLLOWER
I20260808 22:29:23.536181 29537 consensus_queue.cc:260] T 8c018d6838314fc882dbe6c0d1bdb10a P 27db2e9af5d04eb2a695be5342e01774 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e49df4478baa46cbaf98a2b1c1332060" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36615 } } peers { permanent_uuid: "27db2e9af5d04eb2a695be5342e01774" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 34481 } } peers { permanent_uuid: "d84376118b0d44df94afa76d495ae676" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43769 } }
I20260808 22:29:23.536299 29538 consensus_queue.cc:260] T 8c018d6838314fc882dbe6c0d1bdb10a P e49df4478baa46cbaf98a2b1c1332060 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e49df4478baa46cbaf98a2b1c1332060" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36615 } } peers { permanent_uuid: "27db2e9af5d04eb2a695be5342e01774" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 34481 } } peers { permanent_uuid: "d84376118b0d44df94afa76d495ae676" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43769 } }
I20260808 22:29:23.536888 29307 heartbeater.cc:503] Master 127.5.11.254:46669 was elected leader, sending a full tablet report...
I20260808 22:29:23.537293 29538 ts_tablet_manager.cc:1428] T 8c018d6838314fc882dbe6c0d1bdb10a P e49df4478baa46cbaf98a2b1c1332060: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260808 22:29:23.537693 29520 heartbeater.cc:503] Master 127.5.11.254:46669 was elected leader, sending a full tablet report...
I20260808 22:29:23.538203 29537 ts_tablet_manager.cc:1428] T 8c018d6838314fc882dbe6c0d1bdb10a P 27db2e9af5d04eb2a695be5342e01774: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260808 22:29:23.581197 29102 tablet.cc:2369] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260808 22:29:23.715973 29521 tablet.cc:2369] T 8c018d6838314fc882dbe6c0d1bdb10a P e49df4478baa46cbaf98a2b1c1332060: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:29:23.763799 29111 raft_consensus.cc:482] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:29:23.764063 29111 raft_consensus.cc:504] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e49df4478baa46cbaf98a2b1c1332060" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36615 } } peers { permanent_uuid: "27db2e9af5d04eb2a695be5342e01774" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 34481 } } peers { permanent_uuid: "d84376118b0d44df94afa76d495ae676" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43769 } }
I20260808 22:29:23.765328 29111 leader_election.cc:288] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e49df4478baa46cbaf98a2b1c1332060 (127.5.11.195:36615), 27db2e9af5d04eb2a695be5342e01774 (127.5.11.194:34481)
I20260808 22:29:23.770879 29532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:23.766100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:55013 (local address 127.5.11.194:34481)
0808 22:29:23.766234 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:23.766237 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:23.766356 (+ 119us) server_negotiation.cc:407] Connection header received
0808 22:29:23.766514 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:23.766516 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:23.766567 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:23.766642 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:29:23.766889 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:23.767660 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:23.769545 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:23.770154 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:23.770280 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:23.770335 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:23.770651 (+ 316us) server_negotiation.cc:299] Negotiation successful
0808 22:29:23.770741 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 22:29:23.770921 29535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:23.765703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:43105 (local address 127.5.11.195:36615)
0808 22:29:23.765858 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:23.765863 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:23.765963 (+ 100us) server_negotiation.cc:407] Connection header received
0808 22:29:23.766341 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:23.766347 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:23.766415 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:23.766517 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:29:23.766863 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:23.767658 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:23.769545 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:23.770154 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:23.770280 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:23.770335 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:23.770693 (+ 358us) server_negotiation.cc:299] Negotiation successful
0808 22:29:23.770775 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 22:29:23.770949 29545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:23.765997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:34481 (local address 127.5.11.193:55013)
0808 22:29:23.766291 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:23.766303 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:23.766387 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:23.766668 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:23.766671 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:23.766685 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:29:23.766757 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:23.766782 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:23.767809 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:23.767814 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:23.769403 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:23.769416 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:23.770312 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:23.770320 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:23.770429 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:23.770516 (+ 87us) client_negotiation.cc:769] Sending connection context
0808 22:29:23.770598 (+ 82us) client_negotiation.cc:240] Negotiation successful
0808 22:29:23.770735 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":178,"thread_start_us":100,"threads_started":1}
I20260808 22:29:23.770908 29543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:23.765523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:36615 (local address 127.5.11.193:43105)
0808 22:29:23.765865 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:23.765883 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:23.766043 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:23.766553 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:23.766555 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:23.766570 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:29:23.766713 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:23.766720 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:23.767809 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:23.767814 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:23.769403 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:23.769416 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:23.770419 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:23.770422 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:23.770533 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:23.770552 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 22:29:23.770651 (+ 99us) client_negotiation.cc:240] Negotiation successful
0808 22:29:23.770764 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":158,"thread_start_us":120,"threads_started":1}
I20260808 22:29:23.771807 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c018d6838314fc882dbe6c0d1bdb10a" candidate_uuid: "d84376118b0d44df94afa76d495ae676" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e49df4478baa46cbaf98a2b1c1332060" is_pre_election: true
I20260808 22:29:23.771790 29230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c018d6838314fc882dbe6c0d1bdb10a" candidate_uuid: "d84376118b0d44df94afa76d495ae676" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27db2e9af5d04eb2a695be5342e01774" is_pre_election: true
I20260808 22:29:23.772127 29444 raft_consensus.cc:2454] T 8c018d6838314fc882dbe6c0d1bdb10a P e49df4478baa46cbaf98a2b1c1332060 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d84376118b0d44df94afa76d495ae676 in term 0.
I20260808 22:29:23.772140 29230 raft_consensus.cc:2454] T 8c018d6838314fc882dbe6c0d1bdb10a P 27db2e9af5d04eb2a695be5342e01774 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d84376118b0d44df94afa76d495ae676 in term 0.
I20260808 22:29:23.772599 28926 leader_election.cc:302] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 27db2e9af5d04eb2a695be5342e01774, d84376118b0d44df94afa76d495ae676; no voters:
I20260808 22:29:23.772895 29111 raft_consensus.cc:2781] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:29:23.772979 29111 raft_consensus.cc:482] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:29:23.773048 29111 raft_consensus.cc:3037] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:29:23.774295 29111 raft_consensus.cc:504] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e49df4478baa46cbaf98a2b1c1332060" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36615 } } peers { permanent_uuid: "27db2e9af5d04eb2a695be5342e01774" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 34481 } } peers { permanent_uuid: "d84376118b0d44df94afa76d495ae676" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43769 } }
I20260808 22:29:23.774775 29111 leader_election.cc:288] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [CANDIDATE]: Term 1 election: Requested vote from peers e49df4478baa46cbaf98a2b1c1332060 (127.5.11.195:36615), 27db2e9af5d04eb2a695be5342e01774 (127.5.11.194:34481)
I20260808 22:29:23.775139 29230 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c018d6838314fc882dbe6c0d1bdb10a" candidate_uuid: "d84376118b0d44df94afa76d495ae676" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "27db2e9af5d04eb2a695be5342e01774"
I20260808 22:29:23.775142 29444 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8c018d6838314fc882dbe6c0d1bdb10a" candidate_uuid: "d84376118b0d44df94afa76d495ae676" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e49df4478baa46cbaf98a2b1c1332060"
I20260808 22:29:23.775274 29230 raft_consensus.cc:3037] T 8c018d6838314fc882dbe6c0d1bdb10a P 27db2e9af5d04eb2a695be5342e01774 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:29:23.775280 29444 raft_consensus.cc:3037] T 8c018d6838314fc882dbe6c0d1bdb10a P e49df4478baa46cbaf98a2b1c1332060 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:29:23.776367 29444 raft_consensus.cc:2454] T 8c018d6838314fc882dbe6c0d1bdb10a P e49df4478baa46cbaf98a2b1c1332060 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d84376118b0d44df94afa76d495ae676 in term 1.
I20260808 22:29:23.776389 29230 raft_consensus.cc:2454] T 8c018d6838314fc882dbe6c0d1bdb10a P 27db2e9af5d04eb2a695be5342e01774 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d84376118b0d44df94afa76d495ae676 in term 1.
I20260808 22:29:23.776664 28926 leader_election.cc:302] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d84376118b0d44df94afa76d495ae676, e49df4478baa46cbaf98a2b1c1332060; no voters:
I20260808 22:29:23.776849 29111 raft_consensus.cc:2781] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [term 1 FOLLOWER]: Leader election won for term 1
W20260808 22:29:23.777037 29308 tablet.cc:2369] T 8c018d6838314fc882dbe6c0d1bdb10a P 27db2e9af5d04eb2a695be5342e01774: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:29:23.777069 29111 raft_consensus.cc:686] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [term 1 LEADER]: Becoming Leader. State: Replica: d84376118b0d44df94afa76d495ae676, State: Running, Role: LEADER
I20260808 22:29:23.777348 29111 consensus_queue.cc:237] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e49df4478baa46cbaf98a2b1c1332060" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36615 } } peers { permanent_uuid: "27db2e9af5d04eb2a695be5342e01774" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 34481 } } peers { permanent_uuid: "d84376118b0d44df94afa76d495ae676" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43769 } }
I20260808 22:29:23.780434 28816 catalog_manager.cc:5495] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 reported cstate change: term changed from 0 to 1, leader changed from <none> to d84376118b0d44df94afa76d495ae676 (127.5.11.193). New cstate: current_term: 1 leader_uuid: "d84376118b0d44df94afa76d495ae676" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e49df4478baa46cbaf98a2b1c1332060" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36615 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "27db2e9af5d04eb2a695be5342e01774" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 34481 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d84376118b0d44df94afa76d495ae676" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43769 } health_report { overall_health: HEALTHY } } }
I20260808 22:29:23.819228 29549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:23.815799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47650 (local address 127.5.11.193:43769)
0808 22:29:23.816091 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:23.816098 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:23.816110 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:29:23.816185 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:23.816187 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:23.816240 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:23.816339 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:29:23.816567 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:23.817085 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:23.817854 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:23.818083 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:23.818164 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:23.818391 (+ 227us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:29:23.818819 (+ 428us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:29:23.818923 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:23.818971 (+ 48us) server_negotiation.cc:299] Negotiation successful
0808 22:29:23.819018 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":136,"threads_started":1}
I20260808 22:29:23.823295 29444 raft_consensus.cc:1261] T 8c018d6838314fc882dbe6c0d1bdb10a P e49df4478baa46cbaf98a2b1c1332060 [term 1 FOLLOWER]: Refusing update from remote peer d84376118b0d44df94afa76d495ae676: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:29:23.823308 29230 raft_consensus.cc:1261] T 8c018d6838314fc882dbe6c0d1bdb10a P 27db2e9af5d04eb2a695be5342e01774 [term 1 FOLLOWER]: Refusing update from remote peer d84376118b0d44df94afa76d495ae676: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:29:23.824069 29111 consensus_queue.cc:1035] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [LEADER]: Connected to new peer: Peer: permanent_uuid: "27db2e9af5d04eb2a695be5342e01774" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 34481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:29:23.824278 29546 consensus_queue.cc:1035] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e49df4478baa46cbaf98a2b1c1332060" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36615 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:29:23.832420 29551 mvcc.cc:205] Tried to move back new op lower bound from 7316390559013490688 to 7316390558834503680. Current Snapshot: MvccSnapshot[applied={T|T < 7316390559013490688}]
I20260808 22:29:23.833348 29555 mvcc.cc:205] Tried to move back new op lower bound from 7316390559013490688 to 7316390558834503680. Current Snapshot: MvccSnapshot[applied={T|T < 7316390559013490688}]
I20260808 22:29:23.839759 28816 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:39330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:29:23.839932 28816 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:39330:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:29:23.842367 28816 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662: Sending DeleteTablet for 3 replicas of tablet 8c018d6838314fc882dbe6c0d1bdb10a
I20260808 22:29:23.843173 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 28869
I20260808 22:29:23.843140 29199 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c018d6838314fc882dbe6c0d1bdb10a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:29:23 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:45410
I20260808 22:29:23.843267 29412 tablet_service.cc:1514] Processing DeleteTablet for tablet 8c018d6838314fc882dbe6c0d1bdb10a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:29:23 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:51100
I20260808 22:29:23.852509 29568 tablet_replica.cc:330] T 8c018d6838314fc882dbe6c0d1bdb10a P 27db2e9af5d04eb2a695be5342e01774: stopping tablet replica
I20260808 22:29:23.852993 29568 raft_consensus.cc:2229] T 8c018d6838314fc882dbe6c0d1bdb10a P 27db2e9af5d04eb2a695be5342e01774 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:29:23.853434 29568 raft_consensus.cc:2258] T 8c018d6838314fc882dbe6c0d1bdb10a P 27db2e9af5d04eb2a695be5342e01774 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:29:23.855439 29568 ts_tablet_manager.cc:1905] T 8c018d6838314fc882dbe6c0d1bdb10a P 27db2e9af5d04eb2a695be5342e01774: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:29:23.857726 29568 ts_tablet_manager.cc:1918] T 8c018d6838314fc882dbe6c0d1bdb10a P 27db2e9af5d04eb2a695be5342e01774: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:29:23.857811 29568 log.cc:1196] T 8c018d6838314fc882dbe6c0d1bdb10a P 27db2e9af5d04eb2a695be5342e01774: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/wal/wals/8c018d6838314fc882dbe6c0d1bdb10a
I20260808 22:29:23.858196 29568 ts_tablet_manager.cc:1939] T 8c018d6838314fc882dbe6c0d1bdb10a P 27db2e9af5d04eb2a695be5342e01774: Deleting consensus metadata
I20260808 22:29:23.859354 28797 catalog_manager.cc:4841] TS 27db2e9af5d04eb2a695be5342e01774 (127.5.11.194:34481): tablet 8c018d6838314fc882dbe6c0d1bdb10a (table test-table [id=10147643ecad462db6fd371108ab6986]) successfully deleted
I20260808 22:29:23.859416 29569 tablet_replica.cc:330] T 8c018d6838314fc882dbe6c0d1bdb10a P e49df4478baa46cbaf98a2b1c1332060: stopping tablet replica
I20260808 22:29:23.859791 29569 raft_consensus.cc:2229] T 8c018d6838314fc882dbe6c0d1bdb10a P e49df4478baa46cbaf98a2b1c1332060 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:29:23.860103 29569 raft_consensus.cc:2258] T 8c018d6838314fc882dbe6c0d1bdb10a P e49df4478baa46cbaf98a2b1c1332060 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:29:23.861332 29569 ts_tablet_manager.cc:1905] T 8c018d6838314fc882dbe6c0d1bdb10a P e49df4478baa46cbaf98a2b1c1332060: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:29:23.863371 29569 ts_tablet_manager.cc:1918] T 8c018d6838314fc882dbe6c0d1bdb10a P e49df4478baa46cbaf98a2b1c1332060: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:29:23.863512 29569 log.cc:1196] T 8c018d6838314fc882dbe6c0d1bdb10a P e49df4478baa46cbaf98a2b1c1332060: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/wal/wals/8c018d6838314fc882dbe6c0d1bdb10a
I20260808 22:29:23.863909 29569 ts_tablet_manager.cc:1939] T 8c018d6838314fc882dbe6c0d1bdb10a P e49df4478baa46cbaf98a2b1c1332060: Deleting consensus metadata
I20260808 22:29:23.865105 28797 catalog_manager.cc:4841] TS e49df4478baa46cbaf98a2b1c1332060 (127.5.11.195:36615): tablet 8c018d6838314fc882dbe6c0d1bdb10a (table test-table [id=10147643ecad462db6fd371108ab6986]) successfully deleted
W20260808 22:29:23.865267 28794 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.11.193:43769 (error 108)
I20260808 22:29:23.865640 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 29110
I20260808 22:29:23.873855 29531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:23.873526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:43769 (local address 127.0.0.1:47662)
0808 22:29:23.873706 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:23.873774 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.193:43769: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
W20260808 22:29:23.874033 28794 catalog_manager.cc:4584] TS d84376118b0d44df94afa76d495ae676 (127.5.11.193:43769): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8c018d6838314fc882dbe6c0d1bdb10a: Network error: Client connection negotiation failed: client connection to 127.5.11.193:43769: connect: Connection refused (error 111)
I20260808 22:29:23.881923 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 29312
I20260808 22:29:23.899380 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 28769
I20260808 22:29:23.911998 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:46669
--webserver_interface=127.5.11.254
--webserver_port=36703
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:34707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:46669
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:29:24.027951 29572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:29:24.028219 29572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:29:24.028258 29572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:29:24.032332 29572 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:29:24.032387 29572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:29:24.032407 29572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:29:24.032444 29572 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:29:24.032495 29572 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:29:24.037715 29572 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:34707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:46669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:46669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=36703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:29:24.038740 29572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:29:24.039844 29572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:29:24.048292 29582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:24.048286 29578 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:24.048300 29579 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:29:24.048652 29572 server_base.cc:1034] running on GCE node
I20260808 22:29:24.049163 29572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:29:24.050251 29572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:29:24.051442 29572 hybrid_clock.cc:648] HybridClock initialized: now 1786228164051426 us; error 29 us; skew 500 ppm
Aug 08 22:29:24 dist-test-slave-7f01 krb5kdc[27921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228164, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:29:24.054201 29572 init.cc:377] Logged in from keytab as kudu/127.5.11.254@KRBTEST.COM (short username kudu)
I20260808 22:29:24.055044 29572 webserver.cc:460] Webserver started at http://127.5.11.254:36703/ using document root <none> and password file <none>
I20260808 22:29:24.055601 29572 fs_manager.cc:359] Metadata directory not provided
I20260808 22:29:24.055698 29572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:29:24.059459 29572 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260808 22:29:24.061632 29594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:24.062800 29572 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 22:29:24.062943 29572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "ff42823b2ab54e03bac154722af86662"
format_stamp: "Formatted at 2026-08-08 22:29:21 on dist-test-slave-7f01"
server_key: "4fe00a98e8959ff4aae4e018e4c16ce3"
server_key_iv: "bbe40d0c30c37a9cd9f824e048f96b72"
server_key_version: "encryptionkey@0"
I20260808 22:29:24.063498 29572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:29:24.114589 29572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:29:24.117794 29572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:29:24.118028 29572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:29:24.125226 29572 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:46669
I20260808 22:29:24.125531 29672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:46669 every 8 connection(s)
I20260808 22:29:24.126338 29572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:29:24.129488 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 29572
I20260808 22:29:24.129911 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.193:43769
--local_ip_for_outbound_sockets=127.5.11.193
--tserver_master_addrs=127.5.11.254:46669
--webserver_port=40453
--webserver_interface=127.5.11.193
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:34707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:29:24.130759 29673 sys_catalog.cc:263] Verifying existing consensus state
I20260808 22:29:24.132738 29673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662: Bootstrap starting.
I20260808 22:29:24.150306 29673 log.cc:824] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662: Log is configured to *not* fsync() on all Append() calls
I20260808 22:29:24.161343 29673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 22:29:24.161712 29673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662: Bootstrap complete.
I20260808 22:29:24.165808 29673 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff42823b2ab54e03bac154722af86662" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 46669 } }
I20260808 22:29:24.166170 29673 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff42823b2ab54e03bac154722af86662, State: Initialized, Role: FOLLOWER
I20260808 22:29:24.166713 29673 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff42823b2ab54e03bac154722af86662" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 46669 } }
I20260808 22:29:24.166857 29673 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:29:24.166947 29673 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:29:24.167104 29673 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [term 2 FOLLOWER]: Advancing to term 3
I20260808 22:29:24.169116 29673 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff42823b2ab54e03bac154722af86662" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 46669 } }
I20260808 22:29:24.169454 29673 leader_election.cc:302] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff42823b2ab54e03bac154722af86662; no voters:
I20260808 22:29:24.169803 29673 leader_election.cc:288] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 22:29:24.169898 29677 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 22:29:24.170109 29677 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [term 3 LEADER]: Becoming Leader. State: Replica: ff42823b2ab54e03bac154722af86662, State: Running, Role: LEADER
I20260808 22:29:24.170398 29677 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff42823b2ab54e03bac154722af86662" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 46669 } }
I20260808 22:29:24.171026 29673 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:29:24.171830 29678 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "ff42823b2ab54e03bac154722af86662" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff42823b2ab54e03bac154722af86662" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 46669 } } }
I20260808 22:29:24.171967 29678 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [sys.catalog]: This master's current role is: LEADER
I20260808 22:29:24.172439 29679 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff42823b2ab54e03bac154722af86662. Latest consensus state: current_term: 3 leader_uuid: "ff42823b2ab54e03bac154722af86662" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff42823b2ab54e03bac154722af86662" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 46669 } } }
I20260808 22:29:24.172585 29679 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff42823b2ab54e03bac154722af86662 [sys.catalog]: This master's current role is: LEADER
I20260808 22:29:24.172930 29695 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:29:24.178494 29695 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:29:24.179117 29695 catalog_manager.cc:1259] Loaded cluster ID: 1bffbea253484f77945a73d2ea733db3
I20260808 22:29:24.179180 29695 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260808 22:29:24.180523 29695 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fe174fcc9ad google::LogMessage::Fail() at ??:0
@ 0x7fe174fcf88a google::LogMessage::SendToLog() at ??:0
@ 0x7fe174fcc4de google::LogMessage::Flush() at ??:0
@ 0x7fe174fcda19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fe17f90bfdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7fe17f90cd2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fe17f909579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7fe17f94d640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fe17721e3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fe17721ec83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7fe1772203f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7fe17720f512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fe1788536db start_thread at ??:0
@ 0x7fe173aca61f clone at ??:0
W20260808 22:29:24.276312 29675 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:29:24.276558 29675 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:29:24.276602 29675 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:29:24.280581 29675 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:29:24.280633 29675 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:29:24.280714 29675 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.193
I20260808 22:29:24.285878 29675 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:34707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.193:43769
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.11.193
--webserver_port=40453
--tserver_master_addrs=127.5.11.254:46669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29675
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.193
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:29:24.286933 29675 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:29:24.288039 29675 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:29:24.297794 29733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:29:24.297865 29675 server_base.cc:1034] running on GCE node
W20260808 22:29:24.297803 29736 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:24.298081 29738 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:29:24.298633 29675 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:29:24.299392 29675 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:29:24.300607 29675 hybrid_clock.cc:648] HybridClock initialized: now 1786228164300581 us; error 33 us; skew 500 ppm
Aug 08 22:29:24 dist-test-slave-7f01 krb5kdc[27921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228164, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:29:24.304028 29675 init.cc:377] Logged in from keytab as kudu/127.5.11.193@KRBTEST.COM (short username kudu)
I20260808 22:29:24.304980 29675 webserver.cc:460] Webserver started at http://127.5.11.193:40453/ using document root <none> and password file <none>
I20260808 22:29:24.305537 29675 fs_manager.cc:359] Metadata directory not provided
I20260808 22:29:24.305600 29675 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:29:24.309607 29675 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:29:24.311883 29749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:24.313241 29675 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260808 22:29:24.313337 29675 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/wal
uuid: "d84376118b0d44df94afa76d495ae676"
format_stamp: "Formatted at 2026-08-08 22:29:21 on dist-test-slave-7f01"
server_key: "93dc6161b588eb8ca7158c229d8fa5a9"
server_key_iv: "e90ee9ab882aac8153fdc2e267b83b5b"
server_key_version: "encryptionkey@0"
I20260808 22:29:24.313885 29675 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:29:24.328651 29675 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:29:24.332108 29675 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:29:24.332311 29675 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:29:24.332909 29675 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:29:24.334036 29764 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 22:29:24.339674 29675 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 22:29:24.339730 29675 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260808 22:29:24.339841 29675 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 22:29:24.341506 29675 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 22:29:24.341576 29675 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260808 22:29:24.341886 29764 tablet_bootstrap.cc:492] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676: Bootstrap starting.
I20260808 22:29:24.352604 29675 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.193:43769
I20260808 22:29:24.352864 29916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.193:43769 every 8 connection(s)
I20260808 22:29:24.353636 29675 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
I20260808 22:29:24.358239 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 29675
I20260808 22:29:24.358685 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.194:34481
--local_ip_for_outbound_sockets=127.5.11.194
--tserver_master_addrs=127.5.11.254:46669
--webserver_port=37437
--webserver_interface=127.5.11.194
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:34707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:29:24.364065 29764 log.cc:824] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676: Log is configured to *not* fsync() on all Append() calls
I20260808 22:29:24.373350 29764 tablet_bootstrap.cc:492] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 22:29:24.373761 29764 tablet_bootstrap.cc:492] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676: Bootstrap complete.
I20260808 22:29:24.375211 29764 ts_tablet_manager.cc:1397] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676: Time spent bootstrapping tablet: real 0.034s user 0.023s sys 0.006s
I20260808 22:29:24.378477 29764 raft_consensus.cc:348] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e49df4478baa46cbaf98a2b1c1332060" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36615 } } peers { permanent_uuid: "27db2e9af5d04eb2a695be5342e01774" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 34481 } } peers { permanent_uuid: "d84376118b0d44df94afa76d495ae676" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43769 } }
I20260808 22:29:24.379021 29764 raft_consensus.cc:729] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: d84376118b0d44df94afa76d495ae676, State: Initialized, Role: FOLLOWER
I20260808 22:29:24.379559 29764 consensus_queue.cc:260] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e49df4478baa46cbaf98a2b1c1332060" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36615 } } peers { permanent_uuid: "27db2e9af5d04eb2a695be5342e01774" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 34481 } } peers { permanent_uuid: "d84376118b0d44df94afa76d495ae676" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43769 } }
I20260808 22:29:24.380671 29764 ts_tablet_manager.cc:1428] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676: Time spent starting tablet: real 0.005s user 0.008s sys 0.000s
W20260808 22:29:24.397905 29921 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:29:24.355579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.193:42493)
0808 22:29:24.356121 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:24.356174 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:24.357005 (+ 831us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:24.397137 (+ 40132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":198,"threads_started":1}
W20260808 22:29:24.398422 29917 heartbeater.cc:644] Failed to heartbeat to 127.5.11.254:46669 (0 consecutive failures): Network error: Failed to ping master at 127.5.11.254:46669: Client connection negotiation failed: client connection to 127.5.11.254:46669: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 22:29:24.399194 29921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:24.398951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.193:45089)
0808 22:29:24.399078 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:24.399140 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260808 22:29:24.399976 29921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:24.399727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.193:59645)
0808 22:29:24.399888 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:24.399930 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260808 22:29:24.400190 29917 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 22:29:24.479861 29922 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:29:24.480141 29922 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:29:24.480214 29922 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:29:24.484135 29922 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:29:24.484212 29922 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:29:24.484294 29922 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.194
I20260808 22:29:24.489334 29922 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:34707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.194:34481
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.11.194
--webserver_port=37437
--tserver_master_addrs=127.5.11.254:46669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29922
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.194
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:29:24.490314 29922 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:29:24.491356 29922 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:29:24.503479 29935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:24.503494 29941 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:24.503911 29934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:29:24.503908 29922 server_base.cc:1034] running on GCE node
I20260808 22:29:24.504458 29922 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:29:24.505071 29922 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:29:24.506235 29922 hybrid_clock.cc:648] HybridClock initialized: now 1786228164506205 us; error 40 us; skew 500 ppm
Aug 08 22:29:24 dist-test-slave-7f01 krb5kdc[27921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228164, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:29:24.509054 29922 init.cc:377] Logged in from keytab as kudu/127.5.11.194@KRBTEST.COM (short username kudu)
I20260808 22:29:24.509954 29922 webserver.cc:460] Webserver started at http://127.5.11.194:37437/ using document root <none> and password file <none>
I20260808 22:29:24.510515 29922 fs_manager.cc:359] Metadata directory not provided
I20260808 22:29:24.510579 29922 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:29:24.514568 29922 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260808 22:29:24.516651 29950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:24.517947 29922 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:29:24.518079 29922 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/wal
uuid: "27db2e9af5d04eb2a695be5342e01774"
format_stamp: "Formatted at 2026-08-08 22:29:22 on dist-test-slave-7f01"
server_key: "52dedd1382beac0ef83ce7a07dfe95bf"
server_key_iv: "eed92f487c2011f7c09b785ca8f5538b"
server_key_version: "encryptionkey@0"
I20260808 22:29:24.518613 29922 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:29:24.535162 29922 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:29:24.537942 29922 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:29:24.538142 29922 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:29:24.538731 29922 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:29:24.539556 29922 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:29:24.539630 29922 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:24.539695 29922 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:29:24.539767 29922 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:24.548626 29922 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.194:34481
I20260808 22:29:24.548646 30118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.194:34481 every 8 connection(s)
I20260808 22:29:24.549566 29922 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
I20260808 22:29:24.553404 30124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:24.551716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.194:45185)
0808 22:29:24.552623 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:24.552735 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":697,"thread_start_us":540,"threads_started":1}
W20260808 22:29:24.553846 30120 heartbeater.cc:644] Failed to heartbeat to 127.5.11.254:46669 (0 consecutive failures): Network error: Failed to ping master at 127.5.11.254:46669: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
I20260808 22:29:24.554447 30124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:24.554220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.194:48181)
0808 22:29:24.554334 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:24.554386 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260808 22:29:24.555027 30124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:24.554812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.194:46821)
0808 22:29:24.554918 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:24.554965 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 22:29:24.555207 30120 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 22:29:24.556483 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 29922
I20260808 22:29:24.556931 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.195:36615
--local_ip_for_outbound_sockets=127.5.11.195
--tserver_master_addrs=127.5.11.254:46669
--webserver_port=43573
--webserver_interface=127.5.11.195
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:34707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:29:24.669189 30125 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:29:24.669431 30125 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:29:24.669476 30125 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:29:24.673343 30125 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:29:24.673395 30125 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:29:24.673449 30125 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.195
I20260808 22:29:24.678610 30125 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:34707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.195:36615
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.11.195
--webserver_port=43573
--tserver_master_addrs=127.5.11.254:46669
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30125
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.195
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:29:24.679502 30125 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:29:24.680570 30125 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:29:24.690009 30138 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:24.690059 30136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:29:24.690102 30125 server_base.cc:1034] running on GCE node
W20260808 22:29:24.690037 30131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:29:24.690729 30125 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:29:24.691370 30125 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:29:24.692538 30125 hybrid_clock.cc:648] HybridClock initialized: now 1786228164692515 us; error 32 us; skew 500 ppm
Aug 08 22:29:24 dist-test-slave-7f01 krb5kdc[27921](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228164, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:29:24.695281 30125 init.cc:377] Logged in from keytab as kudu/127.5.11.195@KRBTEST.COM (short username kudu)
I20260808 22:29:24.696197 30125 webserver.cc:460] Webserver started at http://127.5.11.195:43573/ using document root <none> and password file <none>
I20260808 22:29:24.696707 30125 fs_manager.cc:359] Metadata directory not provided
I20260808 22:29:24.696765 30125 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:29:24.700374 30125 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:29:24.702311 30156 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:24.703549 30125 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:29:24.703682 30125 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/wal
uuid: "e49df4478baa46cbaf98a2b1c1332060"
format_stamp: "Formatted at 2026-08-08 22:29:22 on dist-test-slave-7f01"
server_key: "b5d4f5261b0ab3ff58fe0ef707124d37"
server_key_iv: "aeb17ba00f9f4fd3223a8e3da3bfe8bc"
server_key_version: "encryptionkey@0"
I20260808 22:29:24.704250 30125 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:29:24.723812 30125 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:29:24.726603 30125 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:29:24.726819 30125 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:29:24.727411 30125 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:29:24.728284 30125 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:29:24.728358 30125 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:24.728432 30125 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:29:24.728477 30125 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:24.737668 30125 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.195:36615
I20260808 22:29:24.737701 30337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.195:36615 every 8 connection(s)
I20260808 22:29:24.738657 30125 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
I20260808 22:29:24.742183 30343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:24.740723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.195:46803)
0808 22:29:24.741382 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:24.741489 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":111,"threads_started":1}
W20260808 22:29:24.742697 30338 heartbeater.cc:644] Failed to heartbeat to 127.5.11.254:46669 (0 consecutive failures): Network error: Failed to ping master at 127.5.11.254:46669: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
I20260808 22:29:24.743232 30343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:24.743025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.195:48061)
0808 22:29:24.743132 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:24.743179 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260808 22:29:24.743906 30343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:24.743639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.195:51675)
0808 22:29:24.743731 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:24.743841 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260808 22:29:24.744103 30338 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 22:29:24.744100 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 30125
I20260808 22:29:25.401322 30345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:25.400843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.193:54833)
0808 22:29:25.401138 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:25.401199 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":80,"threads_started":1}
I20260808 22:29:25.556180 30346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:25.555717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.194:49945)
0808 22:29:25.556029 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:25.556081 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":73,"threads_started":1}
I20260808 22:29:25.745016 30348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:25.744616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.195:56053)
0808 22:29:25.744849 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:25.744903 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":69,"threads_started":1}
I20260808 22:29:26.203682 30350 raft_consensus.cc:482] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:29:26.203900 30350 raft_consensus.cc:504] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e49df4478baa46cbaf98a2b1c1332060" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36615 } } peers { permanent_uuid: "27db2e9af5d04eb2a695be5342e01774" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 34481 } } peers { permanent_uuid: "d84376118b0d44df94afa76d495ae676" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43769 } }
I20260808 22:29:26.205006 30350 leader_election.cc:288] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e49df4478baa46cbaf98a2b1c1332060 (127.5.11.195:36615), 27db2e9af5d04eb2a695be5342e01774 (127.5.11.194:34481)
Aug 08 22:29:26 dist-test-slave-7f01 krb5kdc[27921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228164, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for kudu/127.5.11.195@KRBTEST.COM
Aug 08 22:29:26 dist-test-slave-7f01 krb5kdc[27921](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228164, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for kudu/127.5.11.194@KRBTEST.COM
I20260808 22:29:26.222445 30352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:26.205253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:36615 (local address 127.5.11.193:44215)
0808 22:29:26.205483 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:26.205499 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:26.205607 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:26.207500 (+ 1893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:26.207506 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:26.208058 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:29:26.208572 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:26.208588 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:26.209930 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:26.209933 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:26.211023 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:26.211034 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:26.211763 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:26.211767 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:26.211934 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:26.215421 (+ 3487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:29:26.215452 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:29:26.217194 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:29:26.219345 (+ 2151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:26.219351 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:26.219364 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:26.219733 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:26.220218 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:26.220225 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:26.220230 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:26.220461 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:26.221554 (+ 1093us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:29:26.221564 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:29:26.221745 (+ 181us) client_negotiation.cc:769] Sending connection context
0808 22:29:26.221903 (+ 158us) client_negotiation.cc:240] Negotiation successful
0808 22:29:26.222242 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":85,"spinlock_wait_cycles":13056,"thread_start_us":99,"threads_started":1}
I20260808 22:29:26.222586 30358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:26.205827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:56271 (local address 127.5.11.194:34481)
0808 22:29:26.206104 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:26.206119 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:26.206347 (+ 228us) server_negotiation.cc:407] Connection header received
0808 22:29:26.207084 (+ 737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:26.207090 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:26.207479 (+ 389us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:26.207821 (+ 342us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:26.208765 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:26.209800 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:26.211144 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:26.211605 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:26.211811 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:26.217658 (+ 5847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:26.217683 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:26.217699 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:26.217734 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:26.219690 (+ 1956us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:26.220317 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:26.220323 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:26.220328 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:26.220461 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:26.220819 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:26.220822 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:26.220824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:26.221256 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:26.221368 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:26.222081 (+ 713us) server_negotiation.cc:299] Negotiation successful
0808 22:29:26.222398 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":92,"threads_started":1}
I20260808 22:29:26.222694 30354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:26.205422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:44215 (local address 127.5.11.195:36615)
0808 22:29:26.205917 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:26.205942 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:26.205983 (+ 41us) server_negotiation.cc:407] Connection header received
0808 22:29:26.206681 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:26.206687 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:26.207058 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:26.207433 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:26.208772 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:26.209765 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:26.211146 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:26.211607 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:26.211806 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:26.217444 (+ 5638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:26.217448 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:26.217461 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:26.217490 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:26.219216 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:26.219904 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:26.219911 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:26.219918 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:26.220091 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:26.220597 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:26.220601 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:26.220602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:26.221192 (+ 590us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:26.221348 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:26.222081 (+ 733us) server_negotiation.cc:299] Negotiation successful
0808 22:29:26.222473 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":88,"threads_started":1}
I20260808 22:29:26.223251 30357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:26.205641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:34481 (local address 127.5.11.193:56271)
0808 22:29:26.206231 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:26.206243 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:26.206374 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:26.207710 (+ 1336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:26.207713 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:26.208085 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:29:26.208574 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:26.208589 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:26.209930 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:26.209935 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:26.211016 (+ 1081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:26.211027 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:26.211731 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:26.211757 (+ 26us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:26.211936 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:26.215455 (+ 3519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:29:26.215468 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:29:26.217472 (+ 2004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:29:26.219856 (+ 2384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:26.219859 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:26.219861 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:26.220183 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:26.220584 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:26.220587 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:26.220588 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:26.220704 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:26.221417 (+ 713us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:29:26.221435 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:29:26.221707 (+ 272us) client_negotiation.cc:769] Sending connection context
0808 22:29:26.221935 (+ 228us) client_negotiation.cc:240] Negotiation successful
0808 22:29:26.222343 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"mutex_wait_us":247,"spinlock_wait_cycles":3072,"thread_start_us":78,"threads_started":1}
W20260808 22:29:26.225347 29760 leader_election.cc:341] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e49df4478baa46cbaf98a2b1c1332060 (127.5.11.195:36615): Not found: Tablet not found: 8c018d6838314fc882dbe6c0d1bdb10a
W20260808 22:29:26.225597 29760 leader_election.cc:341] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 27db2e9af5d04eb2a695be5342e01774 (127.5.11.194:34481): Not found: Tablet not found: 8c018d6838314fc882dbe6c0d1bdb10a
I20260808 22:29:26.225672 29760 leader_election.cc:302] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d84376118b0d44df94afa76d495ae676; no voters: 27db2e9af5d04eb2a695be5342e01774, e49df4478baa46cbaf98a2b1c1332060
I20260808 22:29:26.225977 30350 raft_consensus.cc:2726] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 22:29:26.402508 30357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:26.402219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.193:47317)
0808 22:29:26.402358 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:26.402417 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 22:29:26.557225 30359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:26.556833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.194:41675)
0808 22:29:26.557069 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:26.557120 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":78,"threads_started":1}
I20260808 22:29:26.746156 30360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:26.745764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.195:33417)
0808 22:29:26.746010 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:26.746065 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":71,"threads_started":1}
I20260808 22:29:27.403712 30361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:27.403280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.193:48873)
0808 22:29:27.403527 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:27.403580 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":78,"threads_started":1}
I20260808 22:29:27.558614 30363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:27.558170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.194:37153)
0808 22:29:27.558455 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:27.558514 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":102,"threads_started":1}
I20260808 22:29:27.747308 30364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:27.746893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.195:39523)
0808 22:29:27.747152 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:27.747204 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":95,"threads_started":1}
I20260808 22:29:28.405007 30365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:28.404557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.193:59483)
0808 22:29:28.404839 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:28.404904 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":117,"threads_started":1}
I20260808 22:29:28.417013 30368 raft_consensus.cc:482] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:29:28.417137 30368 raft_consensus.cc:504] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e49df4478baa46cbaf98a2b1c1332060" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36615 } } peers { permanent_uuid: "27db2e9af5d04eb2a695be5342e01774" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 34481 } } peers { permanent_uuid: "d84376118b0d44df94afa76d495ae676" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43769 } }
I20260808 22:29:28.417626 30368 leader_election.cc:288] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e49df4478baa46cbaf98a2b1c1332060 (127.5.11.195:36615), 27db2e9af5d04eb2a695be5342e01774 (127.5.11.194:34481)
W20260808 22:29:28.418448 29760 leader_election.cc:341] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e49df4478baa46cbaf98a2b1c1332060 (127.5.11.195:36615): Not found: Tablet not found: 8c018d6838314fc882dbe6c0d1bdb10a
W20260808 22:29:28.418576 29760 leader_election.cc:341] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 27db2e9af5d04eb2a695be5342e01774 (127.5.11.194:34481): Not found: Tablet not found: 8c018d6838314fc882dbe6c0d1bdb10a
I20260808 22:29:28.418615 29760 leader_election.cc:302] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d84376118b0d44df94afa76d495ae676; no voters: 27db2e9af5d04eb2a695be5342e01774, e49df4478baa46cbaf98a2b1c1332060
I20260808 22:29:28.418746 30368 raft_consensus.cc:2726] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 22:29:28.559718 30369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:28.559279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.194:57113)
0808 22:29:28.559550 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:28.559606 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":77,"threads_started":1}
I20260808 22:29:28.748427 30370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:28.748041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.195:39231)
0808 22:29:28.748281 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:28.748334 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":75,"threads_started":1}
I20260808 22:29:29.406301 30371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:29.405858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.193:35173)
0808 22:29:29.406146 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:29.406198 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":74,"threads_started":1}
I20260808 22:29:29.560968 30372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:29.560559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.194:40373)
0808 22:29:29.560807 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:29.560860 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":74,"threads_started":1}
I20260808 22:29:29.749636 30373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:29.749219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.195:47253)
0808 22:29:29.749498 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:29.749549 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":91,"threads_started":1}
I20260808 22:29:30.409098 30374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:30.408631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.193:54433)
0808 22:29:30.408881 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:30.408944 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":91,"threads_started":1}
I20260808 22:29:30.562198 30375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:30.561784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.194:53973)
0808 22:29:30.562041 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:30.562097 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":76,"threads_started":1}
I20260808 22:29:30.750886 30376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:30.750479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.195:36013)
0808 22:29:30.750733 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:30.750788 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":74,"threads_started":1}
I20260808 22:29:31.410636 30379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:31.410069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.193:60453)
0808 22:29:31.410417 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:31.410507 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":101,"threads_started":1}
I20260808 22:29:31.473519 30380 raft_consensus.cc:482] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:29:31.473666 30380 raft_consensus.cc:504] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e49df4478baa46cbaf98a2b1c1332060" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36615 } } peers { permanent_uuid: "27db2e9af5d04eb2a695be5342e01774" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 34481 } } peers { permanent_uuid: "d84376118b0d44df94afa76d495ae676" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43769 } }
I20260808 22:29:31.474093 30380 leader_election.cc:288] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e49df4478baa46cbaf98a2b1c1332060 (127.5.11.195:36615), 27db2e9af5d04eb2a695be5342e01774 (127.5.11.194:34481)
W20260808 22:29:31.474987 29760 leader_election.cc:341] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e49df4478baa46cbaf98a2b1c1332060 (127.5.11.195:36615): Not found: Tablet not found: 8c018d6838314fc882dbe6c0d1bdb10a
W20260808 22:29:31.475121 29760 leader_election.cc:341] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 27db2e9af5d04eb2a695be5342e01774 (127.5.11.194:34481): Not found: Tablet not found: 8c018d6838314fc882dbe6c0d1bdb10a
I20260808 22:29:31.475181 29760 leader_election.cc:302] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d84376118b0d44df94afa76d495ae676; no voters: 27db2e9af5d04eb2a695be5342e01774, e49df4478baa46cbaf98a2b1c1332060
I20260808 22:29:31.475361 30380 raft_consensus.cc:2726] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 22:29:31.563377 30385 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:31.562910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.194:43703)
0808 22:29:31.563230 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:31.563283 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":100,"threads_started":1}
I20260808 22:29:31.752162 30387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:31.751641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.195:41811)
0808 22:29:31.752012 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:31.752068 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":146,"threads_started":1}
I20260808 22:29:32.411836 30389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:32.411355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.193:37993)
0808 22:29:32.411623 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:32.411676 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":102,"threads_started":1}
I20260808 22:29:32.564674 30391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:32.564266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.194:56759)
0808 22:29:32.564528 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:32.564580 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":98,"threads_started":1}
I20260808 22:29:32.753319 30395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:32.752914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.195:41161)
0808 22:29:32.753172 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:32.753228 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":76,"threads_started":1}
I20260808 22:29:33.412969 30396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:33.412530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.193:40765)
0808 22:29:33.412810 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:33.412868 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":82,"threads_started":1}
I20260808 22:29:33.565852 30397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:33.565427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.194:55789)
0808 22:29:33.565686 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:33.565747 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":82,"threads_started":1}
I20260808 22:29:33.754431 30399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:33.754043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.195:48407)
0808 22:29:33.754285 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:33.754344 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":78,"threads_started":1}
I20260808 22:29:34.414273 30400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:34.413851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.193:59179)
0808 22:29:34.414111 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:34.414166 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":81,"threads_started":1}
I20260808 22:29:34.567040 30401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:34.566612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.194:41347)
0808 22:29:34.566889 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:34.566946 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":93,"threads_started":1}
I20260808 22:29:34.755558 30402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:34.755154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.195:47581)
0808 22:29:34.755415 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:34.755470 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":70,"threads_started":1}
I20260808 22:29:35.375916 30403 raft_consensus.cc:482] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:29:35.376055 30403 raft_consensus.cc:504] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e49df4478baa46cbaf98a2b1c1332060" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36615 } } peers { permanent_uuid: "27db2e9af5d04eb2a695be5342e01774" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 34481 } } peers { permanent_uuid: "d84376118b0d44df94afa76d495ae676" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43769 } }
I20260808 22:29:35.376425 30403 leader_election.cc:288] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e49df4478baa46cbaf98a2b1c1332060 (127.5.11.195:36615), 27db2e9af5d04eb2a695be5342e01774 (127.5.11.194:34481)
W20260808 22:29:35.377259 29760 leader_election.cc:341] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e49df4478baa46cbaf98a2b1c1332060 (127.5.11.195:36615): Not found: Tablet not found: 8c018d6838314fc882dbe6c0d1bdb10a
W20260808 22:29:35.377461 29760 leader_election.cc:341] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 27db2e9af5d04eb2a695be5342e01774 (127.5.11.194:34481): Not found: Tablet not found: 8c018d6838314fc882dbe6c0d1bdb10a
I20260808 22:29:35.377532 29760 leader_election.cc:302] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d84376118b0d44df94afa76d495ae676; no voters: 27db2e9af5d04eb2a695be5342e01774, e49df4478baa46cbaf98a2b1c1332060
I20260808 22:29:35.377715 30403 raft_consensus.cc:2726] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 22:29:35.415428 30404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:35.415001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.193:47753)
0808 22:29:35.415265 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:35.415333 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":81,"threads_started":1}
I20260808 22:29:35.568137 30405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:35.567716 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.194:60325)
0808 22:29:35.567983 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:35.568044 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":68,"threads_started":1}
I20260808 22:29:35.756672 30406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:35.756270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.195:42851)
0808 22:29:35.756529 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:35.756583 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":73,"threads_started":1}
I20260808 22:29:36.416661 30407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:36.416241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.193:58467)
0808 22:29:36.416501 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:36.416558 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":74,"threads_started":1}
I20260808 22:29:36.569383 30411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:36.568952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.194:49045)
0808 22:29:36.569232 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:36.569285 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":107,"threads_started":1}
I20260808 22:29:36.757786 30412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:36.757395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.195:33033)
0808 22:29:36.757640 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:36.757694 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":75,"threads_started":1}
I20260808 22:29:37.417873 30413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:37.417416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.193:43491)
0808 22:29:37.417717 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:37.417772 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":79,"threads_started":1}
I20260808 22:29:37.570560 30414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:37.570130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.194:42143)
0808 22:29:37.570386 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:37.570459 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":79,"threads_started":1}
I20260808 22:29:37.758950 30416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:37.758562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.195:60637)
0808 22:29:37.758802 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:37.758858 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":72,"threads_started":1}
I20260808 22:29:38.419238 30422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:38.418677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.193:58495)
0808 22:29:38.419055 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:38.419123 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":100,"threads_started":1}
I20260808 22:29:38.571679 30424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:38.571263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.194:45173)
0808 22:29:38.571525 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:38.571579 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":74,"threads_started":1}
I20260808 22:29:38.760160 30425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:38.759666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.195:54803)
0808 22:29:38.760009 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:38.760065 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":145,"threads_started":1}
I20260808 22:29:39.366205 30426 raft_consensus.cc:482] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:29:39.366339 30426 raft_consensus.cc:504] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e49df4478baa46cbaf98a2b1c1332060" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36615 } } peers { permanent_uuid: "27db2e9af5d04eb2a695be5342e01774" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 34481 } } peers { permanent_uuid: "d84376118b0d44df94afa76d495ae676" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43769 } }
I20260808 22:29:39.366859 30426 leader_election.cc:288] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e49df4478baa46cbaf98a2b1c1332060 (127.5.11.195:36615), 27db2e9af5d04eb2a695be5342e01774 (127.5.11.194:34481)
W20260808 22:29:39.367713 29760 leader_election.cc:341] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 27db2e9af5d04eb2a695be5342e01774 (127.5.11.194:34481): Not found: Tablet not found: 8c018d6838314fc882dbe6c0d1bdb10a
W20260808 22:29:39.367935 29760 leader_election.cc:341] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e49df4478baa46cbaf98a2b1c1332060 (127.5.11.195:36615): Not found: Tablet not found: 8c018d6838314fc882dbe6c0d1bdb10a
I20260808 22:29:39.368011 29760 leader_election.cc:302] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: d84376118b0d44df94afa76d495ae676; no voters: 27db2e9af5d04eb2a695be5342e01774, e49df4478baa46cbaf98a2b1c1332060
I20260808 22:29:39.368160 30426 raft_consensus.cc:2726] T 8c018d6838314fc882dbe6c0d1bdb10a P d84376118b0d44df94afa76d495ae676 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 22:29:39.420608 30427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:39.420214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.193:55351)
0808 22:29:39.420452 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:39.420508 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":74,"threads_started":1}
I20260808 22:29:39.572959 30429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:39.572568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.194:37673)
0808 22:29:39.572810 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:39.572863 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":76,"threads_started":1}
I20260808 22:29:39.753108 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 29675
I20260808 22:29:39.768839 30430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:39.766984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:46669 (local address 127.5.11.195:57007)
0808 22:29:39.768614 (+ 1630us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:39.768691 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:46669: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1508,"thread_start_us":306,"threads_started":1}
I20260808 22:29:39.775250 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 29922
I20260808 22:29:39.784961 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 30125
2026-08-08T22:29:39Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19654 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:29:39 dist-test-slave-7f01 krb5kdc[30435](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:29:39 dist-test-slave-7f01 krb5kdc[30435](info): set up 2 sockets
Aug 08 22:29:39 dist-test-slave-7f01 krb5kdc[30435](info): commencing operation
krb5kdc: starting...
W20260808 22:29:41.066237 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.253s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:29:41 dist-test-slave-7f01 krb5kdc[30435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:29:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:29:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.11.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
I20260808 22:29:41.190996 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:40733
--webserver_interface=127.5.11.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:44513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:40733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:29:41.304421 30452 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:29:41.304709 30452 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:29:41.304796 30452 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:29:41.308887 30452 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 22:29:41.308956 30452 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 22:29:41.308983 30452 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 22:29:41.309026 30452 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:29:41.309067 30452 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:29:41.309103 30452 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:29:41.309154 30452 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:29:41.309192 30452 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:29:41.314306 30452 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:44513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:40733
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:40733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30452
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:29:41.315388 30452 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:29:41.316586 30452 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:29:41.324910 30459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:41.324950 30462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:41.324952 30458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:29:41.325158 30452 server_base.cc:1034] running on GCE node
I20260808 22:29:41.325817 30452 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:29:41.326983 30452 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:29:41.328148 30452 hybrid_clock.cc:648] HybridClock initialized: now 1786228181328115 us; error 43 us; skew 500 ppm
Aug 08 22:29:41 dist-test-slave-7f01 krb5kdc[30435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228181, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:29:41.330850 30452 init.cc:377] Logged in from keytab as kudu/127.5.11.254@KRBTEST.COM (short username kudu)
I20260808 22:29:41.331715 30452 webserver.cc:460] Webserver started at http://127.5.11.254:36593/ using document root <none> and password file <none>
I20260808 22:29:41.332324 30452 fs_manager.cc:359] Metadata directory not provided
I20260808 22:29:41.332422 30452 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:29:41.332669 30452 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:29:41.334661 30452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "46ce689329e247e496c3f4fc2ddc42c8"
format_stamp: "Formatted at 2026-08-08 22:29:41 on dist-test-slave-7f01"
server_key: "59639cb1a4bf84aaab401c18376bc215"
server_key_iv: "69460f7363d9b53345ebba7c280c15d2"
server_key_version: "encryptionkey@0"
I20260808 22:29:41.335237 30452 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "46ce689329e247e496c3f4fc2ddc42c8"
format_stamp: "Formatted at 2026-08-08 22:29:41 on dist-test-slave-7f01"
server_key: "59639cb1a4bf84aaab401c18376bc215"
server_key_iv: "69460f7363d9b53345ebba7c280c15d2"
server_key_version: "encryptionkey@0"
I20260808 22:29:41.338838 30452 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260808 22:29:41.341216 30470 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:41.342327 30452 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:29:41.342476 30452 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "46ce689329e247e496c3f4fc2ddc42c8"
format_stamp: "Formatted at 2026-08-08 22:29:41 on dist-test-slave-7f01"
server_key: "59639cb1a4bf84aaab401c18376bc215"
server_key_iv: "69460f7363d9b53345ebba7c280c15d2"
server_key_version: "encryptionkey@0"
I20260808 22:29:41.342584 30452 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:29:41.359242 30452 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:29:41.361946 30452 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:29:41.362146 30452 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:29:41.369215 30452 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:40733
I20260808 22:29:41.369506 30557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:40733 every 8 connection(s)
I20260808 22:29:41.370157 30452 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:29:41.372954 30558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:29:41.377398 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 30452
I20260808 22:29:41.377502 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:29:41.377794 5167 external_mini_cluster.cc:1428] Setting key 7349b69b8e95ae80816a36321d41e83f
I20260808 22:29:41.378476 30558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8: Bootstrap starting.
I20260808 22:29:41.380856 30558 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8: Neither blocks nor log segments found. Creating new log.
I20260808 22:29:41.381623 30558 log.cc:824] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8: Log is configured to *not* fsync() on all Append() calls
I20260808 22:29:41.383457 30558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8: No bootstrap required, opened a new log
Aug 08 22:29:41 dist-test-slave-7f01 krb5kdc[30435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:29:41.387444 30558 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ce689329e247e496c3f4fc2ddc42c8" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 40733 } }
I20260808 22:29:41.387728 30558 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:29:41.387818 30558 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46ce689329e247e496c3f4fc2ddc42c8, State: Initialized, Role: FOLLOWER
I20260808 22:29:41.388310 30558 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ce689329e247e496c3f4fc2ddc42c8" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 40733 } }
I20260808 22:29:41.388465 30558 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:29:41.388561 30558 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:29:41.388682 30558 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:29:41.389503 30558 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ce689329e247e496c3f4fc2ddc42c8" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 40733 } }
I20260808 22:29:41.389842 30558 leader_election.cc:302] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46ce689329e247e496c3f4fc2ddc42c8; no voters:
I20260808 22:29:41.390174 30558 leader_election.cc:288] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:29:41.390269 30568 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:29:41.390488 30568 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [term 1 LEADER]: Becoming Leader. State: Replica: 46ce689329e247e496c3f4fc2ddc42c8, State: Running, Role: LEADER
I20260808 22:29:41.390841 30568 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ce689329e247e496c3f4fc2ddc42c8" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 40733 } }
I20260808 22:29:41.391441 30558 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:29:41.392553 30570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46ce689329e247e496c3f4fc2ddc42c8. Latest consensus state: current_term: 1 leader_uuid: "46ce689329e247e496c3f4fc2ddc42c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ce689329e247e496c3f4fc2ddc42c8" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 40733 } } }
I20260808 22:29:41.392669 30570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [sys.catalog]: This master's current role is: LEADER
I20260808 22:29:41.393016 30564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:41.379138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37078 (local address 127.5.11.254:40733)
0808 22:29:41.379577 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:41.379592 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:41.379626 (+ 34us) server_negotiation.cc:407] Connection header received
0808 22:29:41.380291 (+ 665us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:41.380308 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:41.380628 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:41.380948 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:41.381771 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:41.382906 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:41.383890 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:41.384351 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:41.384545 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:41.387277 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:41.387282 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:41.387295 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:41.387320 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:41.389258 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:41.389740 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:41.389748 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:41.389758 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:41.389868 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:41.390166 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:41.390169 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:41.390171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:41.390743 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:41.390954 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:41.391611 (+ 657us) server_negotiation.cc:299] Negotiation successful
0808 22:29:41.391979 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":130,"threads_started":1}
I20260808 22:29:41.393064 30581 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:29:41.393460 30569 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46ce689329e247e496c3f4fc2ddc42c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ce689329e247e496c3f4fc2ddc42c8" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 40733 } } }
I20260808 22:29:41.393589 30569 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [sys.catalog]: This master's current role is: LEADER
I20260808 22:29:41.396832 30581 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:29:41.402261 30581 catalog_manager.cc:1347] Generated new cluster ID: 528252c11a89481295a51276d74ccbff
I20260808 22:29:41.402333 30581 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:29:41.420068 30581 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:29:41.421176 30581 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:29:41.427331 30581 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8: Generated new TSK 0
I20260808 22:29:41.427985 30581 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.11.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab.
Entry for principal kudu/127.5.11.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab.
I20260808 22:29:41.462244 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.193:0
--local_ip_for_outbound_sockets=127.5.11.193
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:40733
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:44513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:29:41.585395 30608 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:29:41.585644 30608 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:29:41.585685 30608 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:29:41.589607 30608 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:29:41.589658 30608 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:29:41.589715 30608 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.193
I20260808 22:29:41.594967 30608 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:44513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:40733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30608
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.193
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:29:41.595913 30608 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:29:41.596858 30608 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:29:41.606400 30613 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:41.606427 30618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:29:41.606604 30608 server_base.cc:1034] running on GCE node
W20260808 22:29:41.606528 30615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:29:41.607084 30608 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:29:41.607715 30608 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:29:41.608955 30608 hybrid_clock.cc:648] HybridClock initialized: now 1786228181608940 us; error 25 us; skew 500 ppm
Aug 08 22:29:41 dist-test-slave-7f01 krb5kdc[30435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228181, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:29:41.611666 30608 init.cc:377] Logged in from keytab as kudu/127.5.11.193@KRBTEST.COM (short username kudu)
I20260808 22:29:41.612625 30608 webserver.cc:460] Webserver started at http://127.5.11.193:33087/ using document root <none> and password file <none>
I20260808 22:29:41.613150 30608 fs_manager.cc:359] Metadata directory not provided
I20260808 22:29:41.613212 30608 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:29:41.613394 30608 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:29:41.615468 30608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/data/instance:
uuid: "5c5e651a448342b39b026e3ad60c0373"
format_stamp: "Formatted at 2026-08-08 22:29:41 on dist-test-slave-7f01"
server_key: "fe6f540a9cf0f9cce39359d3e0e59f76"
server_key_iv: "03bae320e167e2d9d31f092ce6ec4fd1"
server_key_version: "encryptionkey@0"
I20260808 22:29:41.616097 30608 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance:
uuid: "5c5e651a448342b39b026e3ad60c0373"
format_stamp: "Formatted at 2026-08-08 22:29:41 on dist-test-slave-7f01"
server_key: "fe6f540a9cf0f9cce39359d3e0e59f76"
server_key_iv: "03bae320e167e2d9d31f092ce6ec4fd1"
server_key_version: "encryptionkey@0"
I20260808 22:29:41.619948 30608 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260808 22:29:41.622386 30626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:41.623560 30608 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:29:41.623699 30608 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/wal
uuid: "5c5e651a448342b39b026e3ad60c0373"
format_stamp: "Formatted at 2026-08-08 22:29:41 on dist-test-slave-7f01"
server_key: "fe6f540a9cf0f9cce39359d3e0e59f76"
server_key_iv: "03bae320e167e2d9d31f092ce6ec4fd1"
server_key_version: "encryptionkey@0"
I20260808 22:29:41.623870 30608 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:29:41.633173 30608 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:29:41.635921 30608 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:29:41.636114 30608 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:29:41.636688 30608 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:29:41.637521 30608 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:29:41.637596 30608 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:41.637663 30608 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:29:41.637713 30608 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:41.646684 30608 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.193:46537
I20260808 22:29:41.646792 30786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.193:46537 every 8 connection(s)
I20260808 22:29:41.647625 30608 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
I20260808 22:29:41.649570 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 30608
I20260808 22:29:41.649706 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance
I20260808 22:29:41.650081 5167 external_mini_cluster.cc:1428] Setting key d4457e20b6dad3e6c9b973f9cacfb55c
Aug 08 22:29:41 dist-test-slave-7f01 krb5kdc[30435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228181, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.11.194@KRBTEST.COM; defaulting to no policy
I20260808 22:29:41.661845 30564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:41.649411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:57445 (local address 127.5.11.254:40733)
0808 22:29:41.649562 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:41.649566 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:41.650662 (+ 1096us) server_negotiation.cc:407] Connection header received
0808 22:29:41.652142 (+ 1480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:41.652148 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:41.652207 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:41.652306 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:41.653420 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:41.654116 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:41.655322 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:41.655565 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:41.655680 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:41.658298 (+ 2618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:41.658301 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:41.658304 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:41.658326 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:41.659784 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:41.660304 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:41.660308 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:41.660310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:41.660397 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:41.660814 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:41.660817 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:41.660818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:41.661000 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:41.661079 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:41.661619 (+ 540us) server_negotiation.cc:299] Negotiation successful
0808 22:29:41.661724 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260808 22:29:41.662612 30794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:41.649916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.193:57445)
0808 22:29:41.650453 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:41.650533 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:41.651937 (+ 1404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:41.652465 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:41.652472 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:41.652810 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:29:41.653272 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:41.653285 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:41.654260 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:41.654264 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:41.655191 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:41.655202 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:41.655719 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:41.655723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:41.655916 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:41.656599 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:29:41.656615 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:29:41.658127 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:29:41.659904 (+ 1777us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:41.659911 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:41.659925 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:41.660194 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:41.660493 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:41.660499 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:41.660503 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:41.660655 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:41.661108 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:29:41.661114 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:29:41.661338 (+ 224us) client_negotiation.cc:769] Sending connection context
0808 22:29:41.661505 (+ 167us) client_negotiation.cc:240] Negotiation successful
0808 22:29:41.661818 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":154,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.194@KRBTEST.COM" created.
I20260808 22:29:41.663988 30787 heartbeater.cc:347] Connected to a master server at 127.5.11.254:40733
I20260808 22:29:41.664239 30787 heartbeater.cc:464] Registering TS with master...
I20260808 22:29:41.664726 30787 heartbeater.cc:511] Master 127.5.11.254:40733 requested a full tablet report, sending...
I20260808 22:29:41.666299 30499 ts_manager.cc:194] Registered new tserver with Master: 5c5e651a448342b39b026e3ad60c0373 (127.5.11.193:46537)
I20260808 22:29:41.667443 30499 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.193@KRBTEST.COM'} at 127.5.11.193:57445
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab.
Entry for principal kudu/127.5.11.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab.
I20260808 22:29:41.680768 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.194:0
--local_ip_for_outbound_sockets=127.5.11.194
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:40733
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:44513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:29:41.792559 30797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:29:41.792865 30797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:29:41.792912 30797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:29:41.796804 30797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:29:41.796860 30797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:29:41.796916 30797 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.194
I20260808 22:29:41.802044 30797 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:44513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:40733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.194
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:29:41.803208 30797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:29:41.804234 30797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:29:41.813989 30805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:41.814002 30808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:41.814002 30806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:29:41.814230 30797 server_base.cc:1034] running on GCE node
I20260808 22:29:41.814672 30797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:29:41.815274 30797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:29:41.816464 30797 hybrid_clock.cc:648] HybridClock initialized: now 1786228181816447 us; error 33 us; skew 500 ppm
Aug 08 22:29:41 dist-test-slave-7f01 krb5kdc[30435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228181, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:29:41.819276 30797 init.cc:377] Logged in from keytab as kudu/127.5.11.194@KRBTEST.COM (short username kudu)
I20260808 22:29:41.820147 30797 webserver.cc:460] Webserver started at http://127.5.11.194:35203/ using document root <none> and password file <none>
I20260808 22:29:41.820672 30797 fs_manager.cc:359] Metadata directory not provided
I20260808 22:29:41.820731 30797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:29:41.820894 30797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:29:41.822832 30797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/data/instance:
uuid: "d39f7e986b2e406189b91dff41d88bd5"
format_stamp: "Formatted at 2026-08-08 22:29:41 on dist-test-slave-7f01"
server_key: "3c418455c336e53ad4d1b191e1aadf40"
server_key_iv: "822c48bc029124f31c9305113c897a11"
server_key_version: "encryptionkey@0"
I20260808 22:29:41.823335 30797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance:
uuid: "d39f7e986b2e406189b91dff41d88bd5"
format_stamp: "Formatted at 2026-08-08 22:29:41 on dist-test-slave-7f01"
server_key: "3c418455c336e53ad4d1b191e1aadf40"
server_key_iv: "822c48bc029124f31c9305113c897a11"
server_key_version: "encryptionkey@0"
I20260808 22:29:41.827111 30797 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 22:29:41.829489 30821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:41.830663 30797 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:29:41.830802 30797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/wal
uuid: "d39f7e986b2e406189b91dff41d88bd5"
format_stamp: "Formatted at 2026-08-08 22:29:41 on dist-test-slave-7f01"
server_key: "3c418455c336e53ad4d1b191e1aadf40"
server_key_iv: "822c48bc029124f31c9305113c897a11"
server_key_version: "encryptionkey@0"
I20260808 22:29:41.830909 30797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:29:41.855563 30797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:29:41.858714 30797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:29:41.858924 30797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:29:41.859531 30797 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:29:41.860409 30797 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:29:41.860467 30797 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:41.860545 30797 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:29:41.860585 30797 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:41.869807 30797 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.194:35497
I20260808 22:29:41.869835 30991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.194:35497 every 8 connection(s)
I20260808 22:29:41.870699 30797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
I20260808 22:29:41.877799 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 30797
I20260808 22:29:41.877940 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance
I20260808 22:29:41.878279 5167 external_mini_cluster.cc:1428] Setting key 166bae7fe91ccf10fefb9bbbcb80f56a
Aug 08 22:29:41 dist-test-slave-7f01 krb5kdc[30435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228181, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:29:41.886241 30564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:41.872477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:33253 (local address 127.5.11.254:40733)
0808 22:29:41.872629 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:41.872632 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:41.873414 (+ 782us) server_negotiation.cc:407] Connection header received
0808 22:29:41.874676 (+ 1262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:41.874679 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:41.874731 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:41.874806 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:41.876060 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:41.876650 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:41.878011 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:41.878348 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:41.878433 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:41.881884 (+ 3451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:41.881889 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:41.881891 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:41.881921 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:41.883697 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:41.884458 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:41.884463 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:41.884467 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:41.884570 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:41.884938 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:41.884944 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:41.884949 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:41.885218 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:41.885328 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:41.885822 (+ 494us) server_negotiation.cc:299] Negotiation successful
0808 22:29:41.885982 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 22:29:41.886888 30996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:41.872791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.194:33253)
0808 22:29:41.873249 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:41.873298 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:41.874468 (+ 1170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:41.874979 (+ 511us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:41.874985 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:41.875352 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:29:41.875918 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:41.875935 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:41.876795 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:41.876798 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:41.877832 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:41.877845 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:41.878649 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:41.878653 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:41.878871 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:41.879709 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:29:41.879728 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:29:41.881653 (+ 1925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:29:41.883956 (+ 2303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:41.883962 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:41.883977 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:41.884359 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:41.884692 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:41.884694 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:41.884695 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:41.884834 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:41.885323 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:29:41.885328 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:29:41.885569 (+ 241us) client_negotiation.cc:769] Sending connection context
0808 22:29:41.885730 (+ 161us) client_negotiation.cc:240] Negotiation successful
0808 22:29:41.885984 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":125,"threads_started":1}
I20260808 22:29:41.888230 30992 heartbeater.cc:347] Connected to a master server at 127.5.11.254:40733
I20260808 22:29:41.888491 30992 heartbeater.cc:464] Registering TS with master...
I20260808 22:29:41.888986 30992 heartbeater.cc:511] Master 127.5.11.254:40733 requested a full tablet report, sending...
I20260808 22:29:41.890134 30499 ts_manager.cc:194] Registered new tserver with Master: d39f7e986b2e406189b91dff41d88bd5 (127.5.11.194:35497)
I20260808 22:29:41.890710 30499 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.194@KRBTEST.COM'} at 127.5.11.194:33253
WARNING: no policy specified for kudu/127.5.11.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab.
Entry for principal kudu/127.5.11.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab.
I20260808 22:29:41.910776 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.195:0
--local_ip_for_outbound_sockets=127.5.11.195
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:40733
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:44513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:29:42.025080 31002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:29:42.025336 31002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:29:42.025379 31002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:29:42.029611 31002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:29:42.029667 31002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:29:42.029752 31002 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.195
I20260808 22:29:42.034866 31002 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:44513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:40733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.195
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:29:42.035849 31002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:29:42.036860 31002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:29:42.046183 31015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:29:42.046178 31009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:29:42.046393 31002 server_base.cc:1034] running on GCE node
W20260808 22:29:42.046180 31010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:29:42.047001 31002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:29:42.047591 31002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:29:42.048749 31002 hybrid_clock.cc:648] HybridClock initialized: now 1786228182048742 us; error 30 us; skew 500 ppm
Aug 08 22:29:42 dist-test-slave-7f01 krb5kdc[30435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228182, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:29:42.051441 31002 init.cc:377] Logged in from keytab as kudu/127.5.11.195@KRBTEST.COM (short username kudu)
I20260808 22:29:42.052253 31002 webserver.cc:460] Webserver started at http://127.5.11.195:36883/ using document root <none> and password file <none>
I20260808 22:29:42.052737 31002 fs_manager.cc:359] Metadata directory not provided
I20260808 22:29:42.052793 31002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:29:42.052946 31002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:29:42.054802 31002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/data/instance:
uuid: "a3751d7e254643ecbdf6a6260148bb6c"
format_stamp: "Formatted at 2026-08-08 22:29:42 on dist-test-slave-7f01"
server_key: "8b6aca52aa2e7fb7e7cffd43774121c3"
server_key_iv: "f9a286397a776843f2ae4247923ad1fe"
server_key_version: "encryptionkey@0"
I20260808 22:29:42.055290 31002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance:
uuid: "a3751d7e254643ecbdf6a6260148bb6c"
format_stamp: "Formatted at 2026-08-08 22:29:42 on dist-test-slave-7f01"
server_key: "8b6aca52aa2e7fb7e7cffd43774121c3"
server_key_iv: "f9a286397a776843f2ae4247923ad1fe"
server_key_version: "encryptionkey@0"
I20260808 22:29:42.059501 31002 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 22:29:42.061873 31029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:42.063149 31002 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 22:29:42.063246 31002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/wal
uuid: "a3751d7e254643ecbdf6a6260148bb6c"
format_stamp: "Formatted at 2026-08-08 22:29:42 on dist-test-slave-7f01"
server_key: "8b6aca52aa2e7fb7e7cffd43774121c3"
server_key_iv: "f9a286397a776843f2ae4247923ad1fe"
server_key_version: "encryptionkey@0"
I20260808 22:29:42.063378 31002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:29:42.074347 31002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:29:42.076956 31002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:29:42.077095 31002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:29:42.077585 31002 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:29:42.078305 31002 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:29:42.078351 31002 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:42.078395 31002 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:29:42.078416 31002 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:29:42.086651 31002 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.195:37539
I20260808 22:29:42.086673 31208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.195:37539 every 8 connection(s)
I20260808 22:29:42.087550 31002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
Aug 08 22:29:42 dist-test-slave-7f01 krb5kdc[30435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228182, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:29:42.097792 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 31002
I20260808 22:29:42.097956 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance
I20260808 22:29:42.098345 5167 external_mini_cluster.cc:1428] Setting key a140e0788004559dcde5d7695d6b0be9
I20260808 22:29:42.101065 30564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:42.089299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:34459 (local address 127.5.11.254:40733)
0808 22:29:42.089441 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:42.089445 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:42.090613 (+ 1168us) server_negotiation.cc:407] Connection header received
0808 22:29:42.091592 (+ 979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:42.091596 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:42.091658 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:42.091787 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:42.092984 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:42.093579 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:42.094662 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:42.094882 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:42.094990 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:42.097302 (+ 2312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:42.097305 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:42.097307 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:42.097330 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:42.098968 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:42.099553 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:42.099558 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:42.099560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:42.099643 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:42.100015 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:42.100019 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:42.100020 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:42.100200 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:42.100309 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:42.100779 (+ 470us) server_negotiation.cc:299] Negotiation successful
0808 22:29:42.100897 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 22:29:42.101755 31212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:42.089627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.195:34459)
0808 22:29:42.090460 (+ 833us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:42.090500 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:42.091385 (+ 885us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:42.091961 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:42.091968 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:42.092381 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:29:42.092832 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:42.092845 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:42.093705 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:42.093708 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:42.094538 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:42.094547 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:42.095026 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:42.095030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:42.095188 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:42.095656 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:29:42.095671 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:29:42.097135 (+ 1464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:29:42.099087 (+ 1952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:42.099092 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:42.099106 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:42.099450 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:42.099762 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:29:42.099766 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:29:42.099768 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:29:42.099912 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:29:42.100308 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:29:42.100314 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:29:42.100543 (+ 229us) client_negotiation.cc:769] Sending connection context
0808 22:29:42.100717 (+ 174us) client_negotiation.cc:240] Negotiation successful
0808 22:29:42.101018 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":660,"thread_start_us":533,"threads_started":1}
I20260808 22:29:42.102823 31209 heartbeater.cc:347] Connected to a master server at 127.5.11.254:40733
I20260808 22:29:42.103077 31209 heartbeater.cc:464] Registering TS with master...
I20260808 22:29:42.103547 31209 heartbeater.cc:511] Master 127.5.11.254:40733 requested a full tablet report, sending...
I20260808 22:29:42.104780 30499 ts_manager.cc:194] Registered new tserver with Master: a3751d7e254643ecbdf6a6260148bb6c (127.5.11.195:37539)
I20260808 22:29:42.105370 30499 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.195@KRBTEST.COM'} at 127.5.11.195:34459
I20260808 22:29:42.112354 5167 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:29:42.121209 30564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:42.114331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37082 (local address 127.5.11.254:40733)
0808 22:29:42.114466 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:42.114471 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:42.114681 (+ 210us) server_negotiation.cc:407] Connection header received
0808 22:29:42.114769 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:42.114771 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:42.114807 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:42.114872 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:29:42.115517 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:42.116078 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:42.116896 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:42.117099 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:42.117171 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:42.118376 (+ 1205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:29:42.118378 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:29:42.118380 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:29:42.118401 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:29:42.119859 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:42.120216 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:42.120218 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:42.120219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:42.120293 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:29:42.120518 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:29:42.120520 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:29:42.120521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:29:42.120723 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:29:42.120841 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:42.121021 (+ 180us) server_negotiation.cc:299] Negotiation successful
0808 22:29:42.121104 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260808 22:29:42.123921 30499 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37082:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:29:42.126806 30499 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:29:42.140877 31226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:42.134922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:35497 (local address 127.0.0.1:52270)
0808 22:29:42.135968 (+ 1046us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:42.135983 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:42.136067 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:42.136966 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:42.136969 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:42.137004 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:29:42.137094 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:42.137101 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:42.138008 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:42.138013 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:42.139340 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:42.139354 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:42.140377 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:42.140382 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:42.140497 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:42.140536 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 22:29:42.140641 (+ 105us) client_negotiation.cc:240] Negotiation successful
0808 22:29:42.140746 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":975,"mutex_wait_us":4,"thread_start_us":100,"threads_started":1}
I20260808 22:29:42.140967 31232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:42.135503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52270 (local address 127.5.11.194:35497)
0808 22:29:42.136548 (+ 1045us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:42.136562 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:42.136582 (+ 20us) server_negotiation.cc:407] Connection header received
0808 22:29:42.136648 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:42.136653 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:42.136811 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:42.136986 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:29:42.137195 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:42.137875 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:42.139465 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:42.140271 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:42.140376 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:42.140491 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:42.140698 (+ 207us) server_negotiation.cc:299] Negotiation successful
0808 22:29:42.140815 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":952,"thread_start_us":125,"threads_started":1}
I20260808 22:29:42.141183 31230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:42.134871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58252 (local address 127.5.11.193:46537)
0808 22:29:42.135341 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:42.135356 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:42.135724 (+ 368us) server_negotiation.cc:407] Connection header received
0808 22:29:42.136053 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:42.136061 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:42.136274 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:42.136399 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:29:42.136805 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:42.137688 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:42.139367 (+ 1679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:42.140230 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:42.140355 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:42.140502 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:42.140941 (+ 439us) server_negotiation.cc:299] Negotiation successful
0808 22:29:42.141039 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":155,"threads_started":1}
I20260808 22:29:42.141570 31231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:42.134732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:46537 (local address 127.0.0.1:58252)
0808 22:29:42.135588 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:42.135635 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:42.135854 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:42.136592 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:42.136596 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:42.136614 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:29:42.136692 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:42.136695 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:42.137817 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:42.137823 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:42.139230 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:42.139240 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:42.140572 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:42.140575 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:42.140710 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:42.140735 (+ 25us) client_negotiation.cc:769] Sending connection context
0808 22:29:42.140805 (+ 70us) client_negotiation.cc:240] Negotiation successful
0808 22:29:42.140867 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"mutex_wait_us":84,"thread_start_us":72,"threads_started":1}
I20260808 22:29:42.143049 31227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:42.135135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:37539 (local address 127.0.0.1:54518)
0808 22:29:42.136183 (+ 1048us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:42.136196 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:42.136352 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:42.137847 (+ 1495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:42.137850 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:42.137870 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:29:42.137960 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:42.137966 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:42.139413 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:42.139419 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:42.140186 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:42.140196 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:42.142624 (+ 2428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:42.142634 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:42.142738 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:42.142765 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 22:29:42.142852 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 22:29:42.142918 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":995,"thread_start_us":89,"threads_started":1}
I20260808 22:29:42.143132 31236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:42.135600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54518 (local address 127.5.11.195:37539)
0808 22:29:42.136317 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:42.136327 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:42.136345 (+ 18us) server_negotiation.cc:407] Connection header received
0808 22:29:42.136485 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:42.136490 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:42.136656 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:42.136793 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:29:42.138137 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:42.139252 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:42.140306 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:42.141125 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:42.141244 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:42.141370 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:42.142905 (+ 1535us) server_negotiation.cc:299] Negotiation successful
0808 22:29:42.142998 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":154,"threads_started":1}
I20260808 22:29:42.144124 30895 tablet_service.cc:1467] Processing CreateTablet for tablet ca39d44803f2494986adb3480680e404 (DEFAULT_TABLE table=test-table [id=594fdd76c8a348c48e72545df49de8f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:29:42.144593 30697 tablet_service.cc:1467] Processing CreateTablet for tablet ca39d44803f2494986adb3480680e404 (DEFAULT_TABLE table=test-table [id=594fdd76c8a348c48e72545df49de8f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:29:42.145131 30895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca39d44803f2494986adb3480680e404. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:29:42.145707 30697 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca39d44803f2494986adb3480680e404. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:29:42.145885 31106 tablet_service.cc:1467] Processing CreateTablet for tablet ca39d44803f2494986adb3480680e404 (DEFAULT_TABLE table=test-table [id=594fdd76c8a348c48e72545df49de8f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:29:42.146937 31106 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca39d44803f2494986adb3480680e404. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:29:42.151691 31238 tablet_bootstrap.cc:492] T ca39d44803f2494986adb3480680e404 P d39f7e986b2e406189b91dff41d88bd5: Bootstrap starting.
I20260808 22:29:42.153081 31240 tablet_bootstrap.cc:492] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373: Bootstrap starting.
I20260808 22:29:42.153961 31241 tablet_bootstrap.cc:492] T ca39d44803f2494986adb3480680e404 P a3751d7e254643ecbdf6a6260148bb6c: Bootstrap starting.
I20260808 22:29:42.154460 31238 tablet_bootstrap.cc:654] T ca39d44803f2494986adb3480680e404 P d39f7e986b2e406189b91dff41d88bd5: Neither blocks nor log segments found. Creating new log.
I20260808 22:29:42.155282 31240 tablet_bootstrap.cc:654] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373: Neither blocks nor log segments found. Creating new log.
I20260808 22:29:42.155316 31238 log.cc:824] T ca39d44803f2494986adb3480680e404 P d39f7e986b2e406189b91dff41d88bd5: Log is configured to *not* fsync() on all Append() calls
I20260808 22:29:42.155853 31241 tablet_bootstrap.cc:654] T ca39d44803f2494986adb3480680e404 P a3751d7e254643ecbdf6a6260148bb6c: Neither blocks nor log segments found. Creating new log.
I20260808 22:29:42.156526 31240 log.cc:824] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373: Log is configured to *not* fsync() on all Append() calls
I20260808 22:29:42.157101 31241 log.cc:824] T ca39d44803f2494986adb3480680e404 P a3751d7e254643ecbdf6a6260148bb6c: Log is configured to *not* fsync() on all Append() calls
I20260808 22:29:42.157644 31238 tablet_bootstrap.cc:492] T ca39d44803f2494986adb3480680e404 P d39f7e986b2e406189b91dff41d88bd5: No bootstrap required, opened a new log
I20260808 22:29:42.157835 31238 ts_tablet_manager.cc:1397] T ca39d44803f2494986adb3480680e404 P d39f7e986b2e406189b91dff41d88bd5: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 22:29:42.158473 31240 tablet_bootstrap.cc:492] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373: No bootstrap required, opened a new log
I20260808 22:29:42.158654 31240 ts_tablet_manager.cc:1397] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 22:29:42.158815 31241 tablet_bootstrap.cc:492] T ca39d44803f2494986adb3480680e404 P a3751d7e254643ecbdf6a6260148bb6c: No bootstrap required, opened a new log
I20260808 22:29:42.159019 31241 ts_tablet_manager.cc:1397] T ca39d44803f2494986adb3480680e404 P a3751d7e254643ecbdf6a6260148bb6c: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260808 22:29:42.161710 31241 raft_consensus.cc:348] T ca39d44803f2494986adb3480680e404 P a3751d7e254643ecbdf6a6260148bb6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c5e651a448342b39b026e3ad60c0373" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46537 } } peers { permanent_uuid: "d39f7e986b2e406189b91dff41d88bd5" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35497 } } peers { permanent_uuid: "a3751d7e254643ecbdf6a6260148bb6c" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37539 } }
I20260808 22:29:42.161688 31238 raft_consensus.cc:348] T ca39d44803f2494986adb3480680e404 P d39f7e986b2e406189b91dff41d88bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c5e651a448342b39b026e3ad60c0373" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46537 } } peers { permanent_uuid: "d39f7e986b2e406189b91dff41d88bd5" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35497 } } peers { permanent_uuid: "a3751d7e254643ecbdf6a6260148bb6c" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37539 } }
I20260808 22:29:42.161940 31241 raft_consensus.cc:374] T ca39d44803f2494986adb3480680e404 P a3751d7e254643ecbdf6a6260148bb6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:29:42.161940 31238 raft_consensus.cc:374] T ca39d44803f2494986adb3480680e404 P d39f7e986b2e406189b91dff41d88bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:29:42.161998 31238 raft_consensus.cc:729] T ca39d44803f2494986adb3480680e404 P d39f7e986b2e406189b91dff41d88bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d39f7e986b2e406189b91dff41d88bd5, State: Initialized, Role: FOLLOWER
I20260808 22:29:42.162017 31241 raft_consensus.cc:729] T ca39d44803f2494986adb3480680e404 P a3751d7e254643ecbdf6a6260148bb6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3751d7e254643ecbdf6a6260148bb6c, State: Initialized, Role: FOLLOWER
I20260808 22:29:42.162290 31240 raft_consensus.cc:348] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c5e651a448342b39b026e3ad60c0373" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46537 } } peers { permanent_uuid: "d39f7e986b2e406189b91dff41d88bd5" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35497 } } peers { permanent_uuid: "a3751d7e254643ecbdf6a6260148bb6c" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37539 } }
I20260808 22:29:42.162423 31238 consensus_queue.cc:260] T ca39d44803f2494986adb3480680e404 P d39f7e986b2e406189b91dff41d88bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c5e651a448342b39b026e3ad60c0373" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46537 } } peers { permanent_uuid: "d39f7e986b2e406189b91dff41d88bd5" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35497 } } peers { permanent_uuid: "a3751d7e254643ecbdf6a6260148bb6c" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37539 } }
I20260808 22:29:42.162529 31240 raft_consensus.cc:374] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:29:42.162580 31240 raft_consensus.cc:729] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c5e651a448342b39b026e3ad60c0373, State: Initialized, Role: FOLLOWER
I20260808 22:29:42.162503 31241 consensus_queue.cc:260] T ca39d44803f2494986adb3480680e404 P a3751d7e254643ecbdf6a6260148bb6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c5e651a448342b39b026e3ad60c0373" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46537 } } peers { permanent_uuid: "d39f7e986b2e406189b91dff41d88bd5" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35497 } } peers { permanent_uuid: "a3751d7e254643ecbdf6a6260148bb6c" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37539 } }
I20260808 22:29:42.162997 31240 consensus_queue.cc:260] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c5e651a448342b39b026e3ad60c0373" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46537 } } peers { permanent_uuid: "d39f7e986b2e406189b91dff41d88bd5" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35497 } } peers { permanent_uuid: "a3751d7e254643ecbdf6a6260148bb6c" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37539 } }
I20260808 22:29:42.163498 31241 ts_tablet_manager.cc:1428] T ca39d44803f2494986adb3480680e404 P a3751d7e254643ecbdf6a6260148bb6c: Time spent starting tablet: real 0.004s user 0.000s sys 0.005s
I20260808 22:29:42.163581 31238 ts_tablet_manager.cc:1428] T ca39d44803f2494986adb3480680e404 P d39f7e986b2e406189b91dff41d88bd5: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 22:29:42.163825 31240 ts_tablet_manager.cc:1428] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 22:29:42.163914 31209 heartbeater.cc:503] Master 127.5.11.254:40733 was elected leader, sending a full tablet report...
I20260808 22:29:42.163951 30992 heartbeater.cc:503] Master 127.5.11.254:40733 was elected leader, sending a full tablet report...
I20260808 22:29:42.164814 30787 heartbeater.cc:503] Master 127.5.11.254:40733 was elected leader, sending a full tablet report...
I20260808 22:29:42.249541 31250 raft_consensus.cc:482] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:29:42.249738 31250 raft_consensus.cc:504] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c5e651a448342b39b026e3ad60c0373" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46537 } } peers { permanent_uuid: "d39f7e986b2e406189b91dff41d88bd5" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35497 } } peers { permanent_uuid: "a3751d7e254643ecbdf6a6260148bb6c" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37539 } }
I20260808 22:29:42.250880 31250 leader_election.cc:288] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d39f7e986b2e406189b91dff41d88bd5 (127.5.11.194:35497), a3751d7e254643ecbdf6a6260148bb6c (127.5.11.195:37539)
I20260808 22:29:42.255151 31232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:42.251168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:37525 (local address 127.5.11.194:35497)
0808 22:29:42.251326 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:42.251331 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:42.251650 (+ 319us) server_negotiation.cc:407] Connection header received
0808 22:29:42.251813 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:42.251817 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:42.251878 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:42.251958 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:29:42.252275 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:42.253088 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:42.254165 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:42.254573 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:42.254708 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:42.254765 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:42.254943 (+ 178us) server_negotiation.cc:299] Negotiation successful
0808 22:29:42.255017 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 22:29:42.255167 31251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:42.251051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:35497 (local address 127.5.11.193:37525)
0808 22:29:42.251452 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:42.251520 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:42.251676 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:42.251985 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:42.251988 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:42.252009 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:29:42.252140 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:42.252147 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:42.253196 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:42.253199 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:42.254018 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:42.254027 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:42.254698 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:42.254702 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:42.254798 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:42.254831 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 22:29:42.254918 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 22:29:42.254991 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":132,"threads_started":1}
I20260808 22:29:42.255183 31236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:42.251278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:48961 (local address 127.5.11.195:37539)
0808 22:29:42.251400 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:42.251404 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:42.251662 (+ 258us) server_negotiation.cc:407] Connection header received
0808 22:29:42.251880 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:42.251885 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:42.251947 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:42.252014 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:29:42.252454 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:42.253177 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:42.254207 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:42.254617 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:42.254745 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:42.254795 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:42.254985 (+ 190us) server_negotiation.cc:299] Negotiation successful
0808 22:29:42.255065 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 22:29:42.255198 31256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:42.251130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:37539 (local address 127.5.11.193:48961)
0808 22:29:42.251528 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:42.251541 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:29:42.251743 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:29:42.252110 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:29:42.252113 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:29:42.252138 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:29:42.252261 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:42.252268 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:42.253306 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:42.253310 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:42.254071 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:29:42.254080 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:29:42.254740 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:29:42.254745 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:29:42.254859 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:42.254877 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 22:29:42.254966 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 22:29:42.255034 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":128,"threads_started":1}
I20260808 22:29:42.255883 30928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca39d44803f2494986adb3480680e404" candidate_uuid: "5c5e651a448342b39b026e3ad60c0373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d39f7e986b2e406189b91dff41d88bd5" is_pre_election: true
I20260808 22:29:42.256083 31138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca39d44803f2494986adb3480680e404" candidate_uuid: "5c5e651a448342b39b026e3ad60c0373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3751d7e254643ecbdf6a6260148bb6c" is_pre_election: true
I20260808 22:29:42.256220 30928 raft_consensus.cc:2454] T ca39d44803f2494986adb3480680e404 P d39f7e986b2e406189b91dff41d88bd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c5e651a448342b39b026e3ad60c0373 in term 0.
I20260808 22:29:42.256390 31138 raft_consensus.cc:2454] T ca39d44803f2494986adb3480680e404 P a3751d7e254643ecbdf6a6260148bb6c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c5e651a448342b39b026e3ad60c0373 in term 0.
I20260808 22:29:42.256738 30629 leader_election.cc:302] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c5e651a448342b39b026e3ad60c0373, d39f7e986b2e406189b91dff41d88bd5; no voters:
I20260808 22:29:42.257055 31250 raft_consensus.cc:2781] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:29:42.257151 31250 raft_consensus.cc:482] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:29:42.257196 31250 raft_consensus.cc:3037] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:29:42.258229 31250 raft_consensus.cc:504] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c5e651a448342b39b026e3ad60c0373" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46537 } } peers { permanent_uuid: "d39f7e986b2e406189b91dff41d88bd5" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35497 } } peers { permanent_uuid: "a3751d7e254643ecbdf6a6260148bb6c" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37539 } }
I20260808 22:29:42.258576 31250 leader_election.cc:288] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373 [CANDIDATE]: Term 1 election: Requested vote from peers d39f7e986b2e406189b91dff41d88bd5 (127.5.11.194:35497), a3751d7e254643ecbdf6a6260148bb6c (127.5.11.195:37539)
I20260808 22:29:42.258901 30928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca39d44803f2494986adb3480680e404" candidate_uuid: "5c5e651a448342b39b026e3ad60c0373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d39f7e986b2e406189b91dff41d88bd5"
I20260808 22:29:42.259001 31138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca39d44803f2494986adb3480680e404" candidate_uuid: "5c5e651a448342b39b026e3ad60c0373" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3751d7e254643ecbdf6a6260148bb6c"
I20260808 22:29:42.259030 30928 raft_consensus.cc:3037] T ca39d44803f2494986adb3480680e404 P d39f7e986b2e406189b91dff41d88bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:29:42.259159 31138 raft_consensus.cc:3037] T ca39d44803f2494986adb3480680e404 P a3751d7e254643ecbdf6a6260148bb6c [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:29:42.260154 30928 raft_consensus.cc:2454] T ca39d44803f2494986adb3480680e404 P d39f7e986b2e406189b91dff41d88bd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c5e651a448342b39b026e3ad60c0373 in term 1.
I20260808 22:29:42.260310 31138 raft_consensus.cc:2454] T ca39d44803f2494986adb3480680e404 P a3751d7e254643ecbdf6a6260148bb6c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c5e651a448342b39b026e3ad60c0373 in term 1.
I20260808 22:29:42.260502 30629 leader_election.cc:302] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c5e651a448342b39b026e3ad60c0373, d39f7e986b2e406189b91dff41d88bd5; no voters:
I20260808 22:29:42.260700 31250 raft_consensus.cc:2781] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:29:42.260944 31250 raft_consensus.cc:686] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373 [term 1 LEADER]: Becoming Leader. State: Replica: 5c5e651a448342b39b026e3ad60c0373, State: Running, Role: LEADER
I20260808 22:29:42.261281 31250 consensus_queue.cc:237] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c5e651a448342b39b026e3ad60c0373" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46537 } } peers { permanent_uuid: "d39f7e986b2e406189b91dff41d88bd5" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35497 } } peers { permanent_uuid: "a3751d7e254643ecbdf6a6260148bb6c" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37539 } }
I20260808 22:29:42.264245 30497 catalog_manager.cc:5495] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c5e651a448342b39b026e3ad60c0373 (127.5.11.193). New cstate: current_term: 1 leader_uuid: "5c5e651a448342b39b026e3ad60c0373" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c5e651a448342b39b026e3ad60c0373" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46537 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d39f7e986b2e406189b91dff41d88bd5" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3751d7e254643ecbdf6a6260148bb6c" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37539 } health_report { overall_health: UNKNOWN } } }
I20260808 22:29:42.289660 31230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:42.286579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58256 (local address 127.5.11.193:46537)
0808 22:29:42.286704 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 22:29:42.286708 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:29:42.286720 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:29:42.286797 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:29:42.286799 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:29:42.286850 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:29:42.286944 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:29:42.287159 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:42.287630 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:42.288446 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:29:42.288645 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:29:42.288717 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:29:42.288973 (+ 256us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:29:42.289365 (+ 392us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:29:42.289453 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 22:29:42.289511 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 22:29:42.289558 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 22:29:42.295611 31138 raft_consensus.cc:1261] T ca39d44803f2494986adb3480680e404 P a3751d7e254643ecbdf6a6260148bb6c [term 1 FOLLOWER]: Refusing update from remote peer 5c5e651a448342b39b026e3ad60c0373: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:29:42.295639 30928 raft_consensus.cc:1261] T ca39d44803f2494986adb3480680e404 P d39f7e986b2e406189b91dff41d88bd5 [term 1 FOLLOWER]: Refusing update from remote peer 5c5e651a448342b39b026e3ad60c0373: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:29:42.296356 31258 consensus_queue.cc:1035] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d39f7e986b2e406189b91dff41d88bd5" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:29:42.296535 31250 consensus_queue.cc:1035] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3751d7e254643ecbdf6a6260148bb6c" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37539 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:29:42.304131 31269 mvcc.cc:205] Tried to move back new op lower bound from 7316390634675986432 to 7316390634544644096. Current Snapshot: MvccSnapshot[applied={T|T < 7316390634675986432}]
I20260808 22:29:42.314617 30499 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:29:42.314795 30499 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37082:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:29:42.317236 30499 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8: Sending DeleteTablet for 3 replicas of tablet ca39d44803f2494986adb3480680e404
I20260808 22:29:42.317973 30697 tablet_service.cc:1514] Processing DeleteTablet for tablet ca39d44803f2494986adb3480680e404 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:29:42 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:58252
I20260808 22:29:42.317988 30895 tablet_service.cc:1514] Processing DeleteTablet for tablet ca39d44803f2494986adb3480680e404 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:29:42 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:52270
I20260808 22:29:42.318171 31106 tablet_service.cc:1514] Processing DeleteTablet for tablet ca39d44803f2494986adb3480680e404 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:29:42 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:54518
I20260808 22:29:42.318635 31284 tablet_replica.cc:330] T ca39d44803f2494986adb3480680e404 P a3751d7e254643ecbdf6a6260148bb6c: stopping tablet replica
I20260808 22:29:42.318750 31283 tablet_replica.cc:330] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373: stopping tablet replica
I20260808 22:29:42.318853 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 30608
I20260808 22:29:42.329360 31282 tablet_replica.cc:330] T ca39d44803f2494986adb3480680e404 P d39f7e986b2e406189b91dff41d88bd5: stopping tablet replica
I20260808 22:29:42.329737 31282 raft_consensus.cc:2229] T ca39d44803f2494986adb3480680e404 P d39f7e986b2e406189b91dff41d88bd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:29:42.330070 31282 raft_consensus.cc:2258] T ca39d44803f2494986adb3480680e404 P d39f7e986b2e406189b91dff41d88bd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:29:42.339008 31284 raft_consensus.cc:2229] T ca39d44803f2494986adb3480680e404 P a3751d7e254643ecbdf6a6260148bb6c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:29:42.339339 31282 ts_tablet_manager.cc:1905] T ca39d44803f2494986adb3480680e404 P d39f7e986b2e406189b91dff41d88bd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:29:42.339421 31284 raft_consensus.cc:2258] T ca39d44803f2494986adb3480680e404 P a3751d7e254643ecbdf6a6260148bb6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:29:42.341075 31284 ts_tablet_manager.cc:1905] T ca39d44803f2494986adb3480680e404 P a3751d7e254643ecbdf6a6260148bb6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:29:42.341889 31282 ts_tablet_manager.cc:1918] T ca39d44803f2494986adb3480680e404 P d39f7e986b2e406189b91dff41d88bd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:29:42.342007 31282 log.cc:1196] T ca39d44803f2494986adb3480680e404 P d39f7e986b2e406189b91dff41d88bd5: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/wal/wals/ca39d44803f2494986adb3480680e404
I20260808 22:29:42.342307 31282 ts_tablet_manager.cc:1939] T ca39d44803f2494986adb3480680e404 P d39f7e986b2e406189b91dff41d88bd5: Deleting consensus metadata
I20260808 22:29:42.343494 30478 catalog_manager.cc:4841] TS d39f7e986b2e406189b91dff41d88bd5 (127.5.11.194:35497): tablet ca39d44803f2494986adb3480680e404 (table test-table [id=594fdd76c8a348c48e72545df49de8f1]) successfully deleted
I20260808 22:29:42.344785 31284 ts_tablet_manager.cc:1918] T ca39d44803f2494986adb3480680e404 P a3751d7e254643ecbdf6a6260148bb6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:29:42.344894 31284 log.cc:1196] T ca39d44803f2494986adb3480680e404 P a3751d7e254643ecbdf6a6260148bb6c: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/wal/wals/ca39d44803f2494986adb3480680e404
I20260808 22:29:42.345306 31284 ts_tablet_manager.cc:1939] T ca39d44803f2494986adb3480680e404 P a3751d7e254643ecbdf6a6260148bb6c: Deleting consensus metadata
I20260808 22:29:42.346661 30479 catalog_manager.cc:4841] TS a3751d7e254643ecbdf6a6260148bb6c (127.5.11.195:37539): tablet ca39d44803f2494986adb3480680e404 (table test-table [id=594fdd76c8a348c48e72545df49de8f1]) successfully deleted
W20260808 22:29:42.347000 30474 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.11.193:46537 (error 108)
I20260808 22:29:42.347364 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 30797
I20260808 22:29:42.356952 31227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:29:42.356523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:46537 (local address 127.0.0.1:58266)
0808 22:29:42.356750 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0808 22:29:42.356832 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.193:46537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
W20260808 22:29:42.357364 30474 catalog_manager.cc:4584] TS 5c5e651a448342b39b026e3ad60c0373 (127.5.11.193:46537): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ca39d44803f2494986adb3480680e404: Network error: Client connection negotiation failed: client connection to 127.5.11.193:46537: connect: Connection refused (error 111)
I20260808 22:29:42.363561 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 31002
I20260808 22:29:42.376543 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 30452
I20260808 22:30:02.390496 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:40733
--webserver_interface=127.5.11.254
--webserver_port=36593
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:44513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:40733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:30:02.504752 31288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:30:02.505000 31288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:30:02.505088 31288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:30:02.508972 31288 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 22:30:02.509014 31288 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 22:30:02.509030 31288 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 22:30:02.509043 31288 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:30:02.509101 31288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:30:02.509132 31288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:30:02.509166 31288 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:30:02.509205 31288 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:30:02.514411 31288 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:44513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:40733
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:40733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=36593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:30:02.515348 31288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:30:02.516341 31288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:30:02.530673 31298 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:02.530668 31302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:02.530712 31288 server_base.cc:1034] running on GCE node
W20260808 22:30:02.530668 31299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:02.531425 31288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:30:02.532553 31288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:30:02.533758 31288 hybrid_clock.cc:648] HybridClock initialized: now 1786228202533738 us; error 30 us; skew 500 ppm
Aug 08 22:30:02 dist-test-slave-7f01 krb5kdc[30435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228202, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:30:02.536550 31288 init.cc:377] Logged in from keytab as kudu/127.5.11.254@KRBTEST.COM (short username kudu)
I20260808 22:30:02.537449 31288 webserver.cc:460] Webserver started at http://127.5.11.254:36593/ using document root <none> and password file <none>
I20260808 22:30:02.538026 31288 fs_manager.cc:359] Metadata directory not provided
I20260808 22:30:02.538137 31288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:30:02.541937 31288 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 22:30:02.544131 31310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:02.545329 31288 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:30:02.545477 31288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "46ce689329e247e496c3f4fc2ddc42c8"
format_stamp: "Formatted at 2026-08-08 22:29:41 on dist-test-slave-7f01"
server_key: "59639cb1a4bf84aaab401c18376bc215"
server_key_iv: "69460f7363d9b53345ebba7c280c15d2"
server_key_version: "encryptionkey@0"
I20260808 22:30:02.546060 31288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:30:02.565469 31288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:30:02.568481 31288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:30:02.568710 31288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:30:02.576429 31288 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:40733
I20260808 22:30:02.576738 31390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:40733 every 8 connection(s)
I20260808 22:30:02.577581 31288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:30:02.577821 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 31288
I20260808 22:30:02.578322 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.193:46537
--local_ip_for_outbound_sockets=127.5.11.193
--tserver_master_addrs=127.5.11.254:40733
--webserver_port=33087
--webserver_interface=127.5.11.193
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:44513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:30:02.582449 31392 sys_catalog.cc:263] Verifying existing consensus state
I20260808 22:30:02.584316 31392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8: Bootstrap starting.
I20260808 22:30:02.602036 31392 log.cc:824] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8: Log is configured to *not* fsync() on all Append() calls
I20260808 22:30:02.610872 31392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 22:30:02.611269 31392 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8: Bootstrap complete.
I20260808 22:30:02.615458 31392 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ce689329e247e496c3f4fc2ddc42c8" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 40733 } }
I20260808 22:30:02.615931 31392 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46ce689329e247e496c3f4fc2ddc42c8, State: Initialized, Role: FOLLOWER
I20260808 22:30:02.616472 31392 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ce689329e247e496c3f4fc2ddc42c8" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 40733 } }
I20260808 22:30:02.616623 31392 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:30:02.616719 31392 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:30:02.616840 31392 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [term 1 FOLLOWER]: Advancing to term 2
I20260808 22:30:02.618649 31392 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ce689329e247e496c3f4fc2ddc42c8" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 40733 } }
I20260808 22:30:02.618989 31392 leader_election.cc:302] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46ce689329e247e496c3f4fc2ddc42c8; no voters:
I20260808 22:30:02.619299 31392 leader_election.cc:288] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260808 22:30:02.619454 31397 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [term 2 FOLLOWER]: Leader election won for term 2
I20260808 22:30:02.619685 31397 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [term 2 LEADER]: Becoming Leader. State: Replica: 46ce689329e247e496c3f4fc2ddc42c8, State: Running, Role: LEADER
I20260808 22:30:02.620064 31397 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ce689329e247e496c3f4fc2ddc42c8" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 40733 } }
I20260808 22:30:02.620532 31392 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:30:02.621696 31398 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "46ce689329e247e496c3f4fc2ddc42c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ce689329e247e496c3f4fc2ddc42c8" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 40733 } } }
I20260808 22:30:02.621845 31398 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [sys.catalog]: This master's current role is: LEADER
I20260808 22:30:02.621701 31400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46ce689329e247e496c3f4fc2ddc42c8. Latest consensus state: current_term: 2 leader_uuid: "46ce689329e247e496c3f4fc2ddc42c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ce689329e247e496c3f4fc2ddc42c8" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 40733 } } }
I20260808 22:30:02.622156 31400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [sys.catalog]: This master's current role is: LEADER
I20260808 22:30:02.622828 31412 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:30:02.627406 31412 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:30:02.628094 31412 catalog_manager.cc:1259] Loaded cluster ID: 528252c11a89481295a51276d74ccbff
I20260808 22:30:02.628158 31412 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:30:02.630604 31412 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:30:02.631314 31412 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8: Loaded TSK: 0
I20260808 22:30:02.648336 31412 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8: Generated new TSK 1
I20260808 22:30:02.648586 31412 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 22:30:02.697319 31395 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:30:02.697592 31395 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:30:02.697665 31395 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:30:02.701625 31395 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:30:02.701710 31395 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:30:02.701803 31395 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.193
I20260808 22:30:02.706987 31395 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:44513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.193:46537
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.11.193
--webserver_port=33087
--tserver_master_addrs=127.5.11.254:40733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31395
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.193
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:30:02.708058 31395 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:30:02.709081 31395 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:30:02.719825 31442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:02.719871 31440 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:02.719892 31439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:02.720418 31395 server_base.cc:1034] running on GCE node
I20260808 22:30:02.720904 31395 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:30:02.721589 31395 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:30:02.722846 31395 hybrid_clock.cc:648] HybridClock initialized: now 1786228202722815 us; error 51 us; skew 500 ppm
Aug 08 22:30:02 dist-test-slave-7f01 krb5kdc[30435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228202, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:30:02.725716 31395 init.cc:377] Logged in from keytab as kudu/127.5.11.193@KRBTEST.COM (short username kudu)
I20260808 22:30:02.726789 31395 webserver.cc:460] Webserver started at http://127.5.11.193:33087/ using document root <none> and password file <none>
I20260808 22:30:02.727367 31395 fs_manager.cc:359] Metadata directory not provided
I20260808 22:30:02.727443 31395 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:30:02.731848 31395 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:30:02.734124 31451 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:02.735397 31395 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:30:02.735492 31395 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/wal
uuid: "5c5e651a448342b39b026e3ad60c0373"
format_stamp: "Formatted at 2026-08-08 22:29:41 on dist-test-slave-7f01"
server_key: "fe6f540a9cf0f9cce39359d3e0e59f76"
server_key_iv: "03bae320e167e2d9d31f092ce6ec4fd1"
server_key_version: "encryptionkey@0"
I20260808 22:30:02.736109 31395 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:30:02.755980 31395 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:30:02.759407 31395 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:30:02.759627 31395 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:30:02.760325 31395 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:30:02.761551 31461 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 22:30:02.765797 31395 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 22:30:02.765864 31395 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260808 22:30:02.765931 31395 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 22:30:02.767695 31395 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 22:30:02.767777 31395 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.002s
I20260808 22:30:02.768054 31461 tablet_bootstrap.cc:492] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373: Bootstrap starting.
I20260808 22:30:02.778999 31395 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.193:46537
I20260808 22:30:02.779088 31616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.193:46537 every 8 connection(s)
I20260808 22:30:02.780242 31395 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
I20260808 22:30:02.787679 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 31395
I20260808 22:30:02.788237 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.194:35497
--local_ip_for_outbound_sockets=127.5.11.194
--tserver_master_addrs=127.5.11.254:40733
--webserver_port=35203
--webserver_interface=127.5.11.194
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:44513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:30:02.789105 31461 log.cc:824] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373: Log is configured to *not* fsync() on all Append() calls
Aug 08 22:30:02 dist-test-slave-7f01 krb5kdc[30435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228202, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:30:02.797889 31461 tablet_bootstrap.cc:492] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 22:30:02.797873 31619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:02.782368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:38011 (local address 127.5.11.254:40733)
0808 22:30:02.782720 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:02.782741 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:02.783053 (+ 312us) server_negotiation.cc:407] Connection header received
0808 22:30:02.784197 (+ 1144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:02.784209 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:02.784507 (+ 298us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:02.784856 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:02.786024 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:02.787002 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:02.788287 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:02.788867 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:02.789047 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:02.791846 (+ 2799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:02.791852 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:02.791864 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:30:02.791894 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:02.794003 (+ 2109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:02.794701 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:02.794708 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:02.794713 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:02.794877 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:02.795324 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:02.795327 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:02.795329 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:02.795818 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:02.795957 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:02.796681 (+ 724us) server_negotiation.cc:299] Negotiation successful
0808 22:30:02.796990 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":117,"threads_started":1}
I20260808 22:30:02.798002 31620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:02.782404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.193:38011)
0808 22:30:02.782897 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:02.782936 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:02.783723 (+ 787us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:02.784919 (+ 1196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:02.784927 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:02.785369 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:30:02.785837 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:02.785850 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:02.787171 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:02.787175 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:02.788160 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:02.788170 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:02.789012 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:02.789015 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:02.789205 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:02.789940 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:30:02.789967 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:30:02.791608 (+ 1641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:30:02.794136 (+ 2528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:30:02.794145 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:30:02.794163 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:30:02.794562 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:30:02.794992 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:30:02.794998 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:30:02.795004 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:30:02.795205 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:30:02.795955 (+ 750us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:30:02.795965 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:30:02.796308 (+ 343us) client_negotiation.cc:769] Sending connection context
0808 22:30:02.796546 (+ 238us) client_negotiation.cc:240] Negotiation successful
0808 22:30:02.796952 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":136,"threads_started":1}
I20260808 22:30:02.798499 31461 tablet_bootstrap.cc:492] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373: Bootstrap complete.
I20260808 22:30:02.799713 31461 ts_tablet_manager.cc:1397] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373: Time spent bootstrapping tablet: real 0.032s user 0.018s sys 0.011s
I20260808 22:30:02.800829 31617 heartbeater.cc:347] Connected to a master server at 127.5.11.254:40733
I20260808 22:30:02.801100 31617 heartbeater.cc:464] Registering TS with master...
I20260808 22:30:02.801720 31617 heartbeater.cc:511] Master 127.5.11.254:40733 requested a full tablet report, sending...
I20260808 22:30:02.802341 31461 raft_consensus.cc:348] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c5e651a448342b39b026e3ad60c0373" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46537 } } peers { permanent_uuid: "d39f7e986b2e406189b91dff41d88bd5" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35497 } } peers { permanent_uuid: "a3751d7e254643ecbdf6a6260148bb6c" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37539 } }
I20260808 22:30:02.802747 31461 raft_consensus.cc:729] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c5e651a448342b39b026e3ad60c0373, State: Initialized, Role: FOLLOWER
I20260808 22:30:02.803277 31461 consensus_queue.cc:260] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5c5e651a448342b39b026e3ad60c0373" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46537 } } peers { permanent_uuid: "d39f7e986b2e406189b91dff41d88bd5" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35497 } } peers { permanent_uuid: "a3751d7e254643ecbdf6a6260148bb6c" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37539 } }
I20260808 22:30:02.804347 31461 ts_tablet_manager.cc:1428] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373: Time spent starting tablet: real 0.005s user 0.003s sys 0.001s
I20260808 22:30:02.804620 31337 ts_manager.cc:194] Registered new tserver with Master: 5c5e651a448342b39b026e3ad60c0373 (127.5.11.193:46537)
I20260808 22:30:02.807961 31337 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.193@KRBTEST.COM'} at 127.5.11.193:38011
I20260808 22:30:02.810432 31617 heartbeater.cc:503] Master 127.5.11.254:40733 was elected leader, sending a full tablet report...
Aug 08 22:30:02 dist-test-slave-7f01 krb5kdc[30435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228202, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for kudu/127.5.11.193@KRBTEST.COM
I20260808 22:30:02.818382 31625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:02.807277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:46537 (local address 127.0.0.1:50828)
0808 22:30:02.808004 (+ 727us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:02.808064 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:02.808267 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:02.808939 (+ 672us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:02.808945 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:02.809360 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:30:02.809491 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:02.809498 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:02.810720 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:02.810725 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:02.811790 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:02.811803 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:02.812379 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:02.812383 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:02.812507 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:02.813088 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:30:02.813105 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:30:02.814701 (+ 1596us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:30:02.816807 (+ 2106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:30:02.816816 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:30:02.816820 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:30:02.817039 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:30:02.817350 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:30:02.817353 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:30:02.817354 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:30:02.817430 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:30:02.817919 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:30:02.817924 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:30:02.818016 (+ 92us) client_negotiation.cc:769] Sending connection context
0808 22:30:02.818124 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 22:30:02.818258 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":108,"threads_started":1}
I20260808 22:30:02.818542 31629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:02.807361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50828 (local address 127.5.11.193:46537)
0808 22:30:02.807694 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:02.807704 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:02.808163 (+ 459us) server_negotiation.cc:407] Connection header received
0808 22:30:02.808434 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:02.808440 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:02.808631 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:02.808767 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:02.809623 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:02.810602 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:02.811901 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:02.812260 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:02.812347 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:02.814845 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:02.814854 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:02.814866 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:30:02.814892 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:02.816696 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:02.817149 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:02.817155 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:02.817157 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:02.817252 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:02.817518 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:02.817521 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:02.817523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:02.817801 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:02.817939 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:02.818230 (+ 291us) server_negotiation.cc:299] Negotiation successful
0808 22:30:02.818391 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":92,"threads_started":1}
I20260808 22:30:02.819515 31522 tablet_service.cc:1514] Processing DeleteTablet for tablet ca39d44803f2494986adb3480680e404 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:29:42 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:50828
I20260808 22:30:02.820387 31630 tablet_replica.cc:330] stopping tablet replica
I20260808 22:30:02.820693 31630 raft_consensus.cc:2229] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:30:02.820958 31630 raft_consensus.cc:2258] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:30:02.821991 31630 ts_tablet_manager.cc:1905] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:30:02.826399 31630 ts_tablet_manager.cc:1918] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:30:02.826500 31630 log.cc:1196] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/wal/wals/ca39d44803f2494986adb3480680e404
I20260808 22:30:02.826790 31630 ts_tablet_manager.cc:1939] T ca39d44803f2494986adb3480680e404 P 5c5e651a448342b39b026e3ad60c0373: Deleting consensus metadata
I20260808 22:30:02.828647 31311 catalog_manager.cc:4841] TS 5c5e651a448342b39b026e3ad60c0373 (127.5.11.193:46537): tablet ca39d44803f2494986adb3480680e404 (table test-table [id=594fdd76c8a348c48e72545df49de8f1]) successfully deleted
W20260808 22:30:02.911964 31623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:30:02.912211 31623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:30:02.912251 31623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:30:02.916617 31623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:30:02.916671 31623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:30:02.916769 31623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.194
I20260808 22:30:02.921929 31623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:44513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.194:35497
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.11.194
--webserver_port=35203
--tserver_master_addrs=127.5.11.254:40733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.194
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:30:02.922916 31623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:30:02.923969 31623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:30:02.933291 31640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:02.933287 31643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:02.933288 31641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:02.933647 31623 server_base.cc:1034] running on GCE node
I20260808 22:30:02.934087 31623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:30:02.934688 31623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:30:02.935858 31623 hybrid_clock.cc:648] HybridClock initialized: now 1786228202935832 us; error 48 us; skew 500 ppm
Aug 08 22:30:02 dist-test-slave-7f01 krb5kdc[30435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228202, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:30:02.938578 31623 init.cc:377] Logged in from keytab as kudu/127.5.11.194@KRBTEST.COM (short username kudu)
I20260808 22:30:02.939498 31623 webserver.cc:460] Webserver started at http://127.5.11.194:35203/ using document root <none> and password file <none>
I20260808 22:30:02.940094 31623 fs_manager.cc:359] Metadata directory not provided
I20260808 22:30:02.940199 31623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:30:02.943845 31623 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 22:30:02.945807 31654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:02.947016 31623 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:30:02.947146 31623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/wal
uuid: "d39f7e986b2e406189b91dff41d88bd5"
format_stamp: "Formatted at 2026-08-08 22:29:41 on dist-test-slave-7f01"
server_key: "3c418455c336e53ad4d1b191e1aadf40"
server_key_iv: "822c48bc029124f31c9305113c897a11"
server_key_version: "encryptionkey@0"
I20260808 22:30:02.947655 31623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:30:02.969580 31623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:30:02.972359 31623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:30:02.972558 31623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:30:02.973112 31623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:30:02.973899 31623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:30:02.973971 31623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:02.974031 31623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:30:02.974073 31623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:02.983063 31623 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.194:35497
I20260808 22:30:02.983089 31835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.194:35497 every 8 connection(s)
I20260808 22:30:02.984088 31623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
I20260808 22:30:02.986842 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 31623
I20260808 22:30:02.987308 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.195:37539
--local_ip_for_outbound_sockets=127.5.11.195
--tserver_master_addrs=127.5.11.254:40733
--webserver_port=36883
--webserver_interface=127.5.11.195
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:44513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 08 22:30:02 dist-test-slave-7f01 krb5kdc[30435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228202, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:30:02.999308 31619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:02.985888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:55847 (local address 127.5.11.254:40733)
0808 22:30:02.986033 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:02.986036 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:02.987337 (+ 1301us) server_negotiation.cc:407] Connection header received
0808 22:30:02.988410 (+ 1073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:02.988417 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:02.988490 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:02.988594 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:02.989727 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:02.990595 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:02.991713 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:02.992193 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:02.992357 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:02.995159 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:02.995165 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:02.995171 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:30:02.995207 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:02.997046 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:02.997551 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:02.997558 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:02.997562 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:02.997675 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:02.998019 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:02.998025 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:02.998030 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:02.998306 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:02.998424 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:02.998906 (+ 482us) server_negotiation.cc:299] Negotiation successful
0808 22:30:02.999067 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 22:30:02.999934 31839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:02.986234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.194:55847)
0808 22:30:02.987152 (+ 918us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:02.987203 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:02.988175 (+ 972us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:02.988746 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:02.988753 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:02.989105 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:30:02.989565 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:02.989578 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:02.990725 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:02.990727 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:02.991579 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:02.991588 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:02.992418 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:02.992423 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:02.992653 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:02.993270 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:30:02.993286 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:30:02.994973 (+ 1687us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:30:02.997170 (+ 2197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:30:02.997175 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:30:02.997186 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:30:02.997444 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:30:02.997784 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:30:02.997787 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:30:02.997788 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:30:02.997920 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:30:02.998423 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:30:02.998428 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:30:02.998651 (+ 223us) client_negotiation.cc:769] Sending connection context
0808 22:30:02.998800 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 22:30:02.999037 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":712,"thread_start_us":575,"threads_started":1}
I20260808 22:30:03.001129 31837 heartbeater.cc:347] Connected to a master server at 127.5.11.254:40733
I20260808 22:30:03.001384 31837 heartbeater.cc:464] Registering TS with master...
I20260808 22:30:03.001870 31837 heartbeater.cc:511] Master 127.5.11.254:40733 requested a full tablet report, sending...
I20260808 22:30:03.002996 31337 ts_manager.cc:194] Registered new tserver with Master: d39f7e986b2e406189b91dff41d88bd5 (127.5.11.194:35497)
I20260808 22:30:03.003563 31337 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.194@KRBTEST.COM'} at 127.5.11.194:55847
W20260808 22:30:03.103639 31840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:30:03.103984 31840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:30:03.104075 31840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:30:03.108063 31840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:30:03.108160 31840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:30:03.108253 31840 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.195
I20260808 22:30:03.113531 31840 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:44513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.195:37539
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.11.195
--webserver_port=36883
--tserver_master_addrs=127.5.11.254:40733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31840
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.195
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:30:03.114585 31840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:30:03.115887 31840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:30:03.125976 31850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:03.126041 31840 server_base.cc:1034] running on GCE node
W20260808 22:30:03.126029 31854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:03.125976 31856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:03.126631 31840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:30:03.127329 31840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:30:03.128497 31840 hybrid_clock.cc:648] HybridClock initialized: now 1786228203128461 us; error 49 us; skew 500 ppm
Aug 08 22:30:03 dist-test-slave-7f01 krb5kdc[30435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228203, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:30:03.131454 31840 init.cc:377] Logged in from keytab as kudu/127.5.11.195@KRBTEST.COM (short username kudu)
I20260808 22:30:03.132367 31840 webserver.cc:460] Webserver started at http://127.5.11.195:36883/ using document root <none> and password file <none>
I20260808 22:30:03.132925 31840 fs_manager.cc:359] Metadata directory not provided
I20260808 22:30:03.133021 31840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:30:03.136878 31840 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.004s
I20260808 22:30:03.138919 31865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:03.140254 31840 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:30:03.140383 31840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/wal
uuid: "a3751d7e254643ecbdf6a6260148bb6c"
format_stamp: "Formatted at 2026-08-08 22:29:42 on dist-test-slave-7f01"
server_key: "8b6aca52aa2e7fb7e7cffd43774121c3"
server_key_iv: "f9a286397a776843f2ae4247923ad1fe"
server_key_version: "encryptionkey@0"
I20260808 22:30:03.140977 31840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:30:03.154224 31840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:30:03.157007 31840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:30:03.157198 31840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:30:03.157763 31840 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:30:03.158600 31840 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:30:03.158648 31840 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:03.158712 31840 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:30:03.158748 31840 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:03.167531 31840 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.195:37539
I20260808 22:30:03.167613 32042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.195:37539 every 8 connection(s)
I20260808 22:30:03.168577 31840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
I20260808 22:30:03.175287 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 31840
Aug 08 22:30:03 dist-test-slave-7f01 krb5kdc[30435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228203, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:30:03.183355 31619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:03.170465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:48879 (local address 127.5.11.254:40733)
0808 22:30:03.170609 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:03.170613 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:03.171953 (+ 1340us) server_negotiation.cc:407] Connection header received
0808 22:30:03.172879 (+ 926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:03.172883 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:03.172939 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:03.173042 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:03.174233 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:03.174761 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:03.176039 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:03.176372 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:03.176484 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:03.179268 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:03.179272 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:03.179274 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:30:03.179303 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:03.181180 (+ 1877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:03.181778 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:03.181785 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:03.181787 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:03.181887 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:03.182241 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:03.182245 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:03.182247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:03.182438 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:03.182552 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:03.183053 (+ 501us) server_negotiation.cc:299] Negotiation successful
0808 22:30:03.183182 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 22:30:03.184358 32046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:03.170860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.195:48879)
0808 22:30:03.171788 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:03.171853 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:03.172670 (+ 817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:03.173227 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:03.173234 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:03.173592 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:30:03.174085 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:03.174098 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:03.174893 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:03.174897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:03.175908 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:03.175919 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:03.176554 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:03.176561 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:03.176731 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:03.177365 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:30:03.177388 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:30:03.179083 (+ 1695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:30:03.181310 (+ 2227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:30:03.181315 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:30:03.181327 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:30:03.181669 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:30:03.181986 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:30:03.181989 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:30:03.181991 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:30:03.182136 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:30:03.182551 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:30:03.182558 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:30:03.182805 (+ 247us) client_negotiation.cc:769] Sending connection context
0808 22:30:03.183004 (+ 199us) client_negotiation.cc:240] Negotiation successful
0808 22:30:03.183338 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":687,"thread_start_us":552,"threads_started":1}
I20260808 22:30:03.184695 32051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:03.176063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33802 (local address 127.5.11.254:40733)
0808 22:30:03.176655 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:03.176658 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:03.176666 (+ 8us) server_negotiation.cc:407] Connection header received
0808 22:30:03.176692 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:03.176694 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:03.176723 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:03.176790 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:03.177723 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:03.178398 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:03.179421 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:03.179721 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:03.179852 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:03.181530 (+ 1678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:03.181533 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:03.181541 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:30:03.181562 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:03.182996 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:03.183424 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:03.183447 (+ 23us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:03.183451 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:03.183541 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:03.183851 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:03.183858 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:03.183860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:03.184028 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:03.184246 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:03.184394 (+ 148us) server_negotiation.cc:299] Negotiation successful
0808 22:30:03.184509 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":115,"threads_started":1}
I20260808 22:30:03.185760 32043 heartbeater.cc:347] Connected to a master server at 127.5.11.254:40733
I20260808 22:30:03.185995 32043 heartbeater.cc:464] Registering TS with master...
I20260808 22:30:03.186493 32043 heartbeater.cc:511] Master 127.5.11.254:40733 requested a full tablet report, sending...
I20260808 22:30:03.187498 31335 ts_manager.cc:194] Registered new tserver with Master: a3751d7e254643ecbdf6a6260148bb6c (127.5.11.195:37539)
I20260808 22:30:03.188112 31335 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.195@KRBTEST.COM'} at 127.5.11.195:48879
I20260808 22:30:03.198570 5167 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:30:03.202565 32051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:03.199244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33814 (local address 127.5.11.254:40733)
0808 22:30:03.199406 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:03.199409 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:03.199583 (+ 174us) server_negotiation.cc:407] Connection header received
0808 22:30:03.199725 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:03.199727 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:03.199887 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:03.199983 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:30:03.200231 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:03.200745 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:03.201566 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:03.201784 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:03.201862 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:03.202122 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:30:03.202256 (+ 134us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0808 22:30:03.202467 (+ 211us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33814: token expired
Metrics: {"server-negotiator.queue_time_us":68}
W20260808 22:30:03.202658 32051 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33814: token expired
W20260808 22:30:03.203008 31213 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.5.11.254:40733)
I20260808 22:30:03.210143 32051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:03.203370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33826 (local address 127.5.11.254:40733)
0808 22:30:03.203480 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:03.203483 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:03.203550 (+ 67us) server_negotiation.cc:407] Connection header received
0808 22:30:03.203620 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:03.203622 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:03.203654 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:03.203714 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:03.204370 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:03.204849 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:03.205592 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:03.205790 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:03.205864 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:03.207166 (+ 1302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:03.207169 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:03.207171 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:30:03.207193 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:03.208773 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:03.209156 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:03.209159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:03.209160 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:03.209238 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:03.209489 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:03.209492 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:03.209493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:03.209648 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:03.209748 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:03.209943 (+ 195us) server_negotiation.cc:299] Negotiation successful
0808 22:30:03.210036 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 22:30:03.212697 31335 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33826:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:30:03.216192 31335 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:30:03.224443 31522 tablet_service.cc:1467] Processing CreateTablet for tablet 97ececcc314a441eb04add447679bfd5 (DEFAULT_TABLE table=test-table [id=36444d5880134ccc976e6cd63d41bf3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:30:03.225219 31522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97ececcc314a441eb04add447679bfd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:30:03.228413 31461 tablet_bootstrap.cc:492] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373: Bootstrap starting.
I20260808 22:30:03.228806 31625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:03.223194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:37539 (local address 127.0.0.1:43256)
0808 22:30:03.223319 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:03.223334 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:03.223435 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:03.224404 (+ 969us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:03.224407 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:03.224428 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:30:03.224552 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:03.224561 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:03.225898 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:03.225902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:03.227042 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:03.227055 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:03.228348 (+ 1293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:03.228351 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:03.228504 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:03.228566 (+ 62us) client_negotiation.cc:769] Sending connection context
0808 22:30:03.228624 (+ 58us) client_negotiation.cc:240] Negotiation successful
0808 22:30:03.228699 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":2}
I20260808 22:30:03.229394 31461 tablet_bootstrap.cc:654] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373: Neither blocks nor log segments found. Creating new log.
I20260808 22:30:03.229475 32059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:03.223415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43256 (local address 127.5.11.195:37539)
0808 22:30:03.223805 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:03.223818 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:03.223850 (+ 32us) server_negotiation.cc:407] Connection header received
0808 22:30:03.223919 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:03.223924 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:03.224172 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:03.224410 (+ 238us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:30:03.224678 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:03.225767 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:03.227200 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:03.228190 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:03.229099 (+ 909us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:03.229217 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:03.229259 (+ 42us) server_negotiation.cc:299] Negotiation successful
0808 22:30:03.229337 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":175,"threads_started":1}
I20260808 22:30:03.229509 32063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:03.223571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:35497 (local address 127.0.0.1:44858)
0808 22:30:03.224080 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:03.224095 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:03.224198 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:03.225202 (+ 1004us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:03.225205 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:03.225221 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:30:03.225336 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:03.225344 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:03.226616 (+ 1272us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:03.226623 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:03.227708 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:03.227723 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:03.229036 (+ 1313us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:03.229040 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:03.229138 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:03.229164 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 22:30:03.229237 (+ 73us) client_negotiation.cc:240] Negotiation successful
0808 22:30:03.229334 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":441,"mutex_wait_us":66,"thread_start_us":97,"threads_started":1}
I20260808 22:30:03.230041 32064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:03.224019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44858 (local address 127.5.11.194:35497)
0808 22:30:03.224490 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:03.224503 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:03.224526 (+ 23us) server_negotiation.cc:407] Connection header received
0808 22:30:03.224603 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:03.224611 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:03.224836 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:03.225012 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:30:03.225472 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:03.226478 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:03.227899 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:03.228703 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:03.228808 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:03.228953 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:03.229750 (+ 797us) server_negotiation.cc:299] Negotiation successful
0808 22:30:03.229874 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":130,"threads_started":1}
I20260808 22:30:03.231094 31461 tablet_bootstrap.cc:492] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373: No bootstrap required, opened a new log
I20260808 22:30:03.231201 31461 ts_tablet_manager.cc:1397] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260808 22:30:03.231603 31461 raft_consensus.cc:348] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3751d7e254643ecbdf6a6260148bb6c" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37539 } } peers { permanent_uuid: "d39f7e986b2e406189b91dff41d88bd5" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35497 } } peers { permanent_uuid: "5c5e651a448342b39b026e3ad60c0373" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46537 } }
I20260808 22:30:03.231727 31461 raft_consensus.cc:374] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:30:03.231805 31461 raft_consensus.cc:729] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c5e651a448342b39b026e3ad60c0373, State: Initialized, Role: FOLLOWER
I20260808 22:30:03.231945 31461 consensus_queue.cc:260] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3751d7e254643ecbdf6a6260148bb6c" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37539 } } peers { permanent_uuid: "d39f7e986b2e406189b91dff41d88bd5" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35497 } } peers { permanent_uuid: "5c5e651a448342b39b026e3ad60c0373" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46537 } }
I20260808 22:30:03.232743 31461 ts_tablet_manager.cc:1428] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260808 22:30:03.232775 31729 tablet_service.cc:1467] Processing CreateTablet for tablet 97ececcc314a441eb04add447679bfd5 (DEFAULT_TABLE table=test-table [id=36444d5880134ccc976e6cd63d41bf3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:30:03.232748 31940 tablet_service.cc:1467] Processing CreateTablet for tablet 97ececcc314a441eb04add447679bfd5 (DEFAULT_TABLE table=test-table [id=36444d5880134ccc976e6cd63d41bf3d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:30:03.233901 31940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97ececcc314a441eb04add447679bfd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:30:03.233898 31729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97ececcc314a441eb04add447679bfd5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:30:03.242226 32066 tablet_bootstrap.cc:492] T 97ececcc314a441eb04add447679bfd5 P a3751d7e254643ecbdf6a6260148bb6c: Bootstrap starting.
I20260808 22:30:03.242226 32067 tablet_bootstrap.cc:492] T 97ececcc314a441eb04add447679bfd5 P d39f7e986b2e406189b91dff41d88bd5: Bootstrap starting.
I20260808 22:30:03.244735 32066 tablet_bootstrap.cc:654] T 97ececcc314a441eb04add447679bfd5 P a3751d7e254643ecbdf6a6260148bb6c: Neither blocks nor log segments found. Creating new log.
I20260808 22:30:03.245507 32067 tablet_bootstrap.cc:654] T 97ececcc314a441eb04add447679bfd5 P d39f7e986b2e406189b91dff41d88bd5: Neither blocks nor log segments found. Creating new log.
I20260808 22:30:03.246126 32066 log.cc:824] T 97ececcc314a441eb04add447679bfd5 P a3751d7e254643ecbdf6a6260148bb6c: Log is configured to *not* fsync() on all Append() calls
I20260808 22:30:03.246274 32067 log.cc:824] T 97ececcc314a441eb04add447679bfd5 P d39f7e986b2e406189b91dff41d88bd5: Log is configured to *not* fsync() on all Append() calls
I20260808 22:30:03.248083 32066 tablet_bootstrap.cc:492] T 97ececcc314a441eb04add447679bfd5 P a3751d7e254643ecbdf6a6260148bb6c: No bootstrap required, opened a new log
I20260808 22:30:03.248121 32067 tablet_bootstrap.cc:492] T 97ececcc314a441eb04add447679bfd5 P d39f7e986b2e406189b91dff41d88bd5: No bootstrap required, opened a new log
I20260808 22:30:03.248323 32067 ts_tablet_manager.cc:1397] T 97ececcc314a441eb04add447679bfd5 P d39f7e986b2e406189b91dff41d88bd5: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 22:30:03.248325 32066 ts_tablet_manager.cc:1397] T 97ececcc314a441eb04add447679bfd5 P a3751d7e254643ecbdf6a6260148bb6c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 22:30:03.251564 32066 raft_consensus.cc:348] T 97ececcc314a441eb04add447679bfd5 P a3751d7e254643ecbdf6a6260148bb6c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3751d7e254643ecbdf6a6260148bb6c" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37539 } } peers { permanent_uuid: "d39f7e986b2e406189b91dff41d88bd5" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35497 } } peers { permanent_uuid: "5c5e651a448342b39b026e3ad60c0373" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46537 } }
I20260808 22:30:03.251564 32067 raft_consensus.cc:348] T 97ececcc314a441eb04add447679bfd5 P d39f7e986b2e406189b91dff41d88bd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3751d7e254643ecbdf6a6260148bb6c" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37539 } } peers { permanent_uuid: "d39f7e986b2e406189b91dff41d88bd5" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35497 } } peers { permanent_uuid: "5c5e651a448342b39b026e3ad60c0373" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46537 } }
I20260808 22:30:03.251857 32067 raft_consensus.cc:374] T 97ececcc314a441eb04add447679bfd5 P d39f7e986b2e406189b91dff41d88bd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:30:03.251840 32066 raft_consensus.cc:374] T 97ececcc314a441eb04add447679bfd5 P a3751d7e254643ecbdf6a6260148bb6c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:30:03.251920 32066 raft_consensus.cc:729] T 97ececcc314a441eb04add447679bfd5 P a3751d7e254643ecbdf6a6260148bb6c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3751d7e254643ecbdf6a6260148bb6c, State: Initialized, Role: FOLLOWER
I20260808 22:30:03.251930 32067 raft_consensus.cc:729] T 97ececcc314a441eb04add447679bfd5 P d39f7e986b2e406189b91dff41d88bd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d39f7e986b2e406189b91dff41d88bd5, State: Initialized, Role: FOLLOWER
I20260808 22:30:03.252360 32066 consensus_queue.cc:260] T 97ececcc314a441eb04add447679bfd5 P a3751d7e254643ecbdf6a6260148bb6c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3751d7e254643ecbdf6a6260148bb6c" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37539 } } peers { permanent_uuid: "d39f7e986b2e406189b91dff41d88bd5" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35497 } } peers { permanent_uuid: "5c5e651a448342b39b026e3ad60c0373" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46537 } }
I20260808 22:30:03.252360 32067 consensus_queue.cc:260] T 97ececcc314a441eb04add447679bfd5 P d39f7e986b2e406189b91dff41d88bd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3751d7e254643ecbdf6a6260148bb6c" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37539 } } peers { permanent_uuid: "d39f7e986b2e406189b91dff41d88bd5" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35497 } } peers { permanent_uuid: "5c5e651a448342b39b026e3ad60c0373" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46537 } }
I20260808 22:30:03.252986 31837 heartbeater.cc:503] Master 127.5.11.254:40733 was elected leader, sending a full tablet report...
I20260808 22:30:03.252986 32043 heartbeater.cc:503] Master 127.5.11.254:40733 was elected leader, sending a full tablet report...
I20260808 22:30:03.253352 32066 ts_tablet_manager.cc:1428] T 97ececcc314a441eb04add447679bfd5 P a3751d7e254643ecbdf6a6260148bb6c: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 22:30:03.253352 32067 ts_tablet_manager.cc:1428] T 97ececcc314a441eb04add447679bfd5 P d39f7e986b2e406189b91dff41d88bd5: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260808 22:30:03.266602 32071 raft_consensus.cc:482] T 97ececcc314a441eb04add447679bfd5 P a3751d7e254643ecbdf6a6260148bb6c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:30:03.266796 32071 raft_consensus.cc:504] T 97ececcc314a441eb04add447679bfd5 P a3751d7e254643ecbdf6a6260148bb6c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3751d7e254643ecbdf6a6260148bb6c" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37539 } } peers { permanent_uuid: "d39f7e986b2e406189b91dff41d88bd5" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35497 } } peers { permanent_uuid: "5c5e651a448342b39b026e3ad60c0373" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46537 } }
I20260808 22:30:03.267931 32071 leader_election.cc:288] T 97ececcc314a441eb04add447679bfd5 P a3751d7e254643ecbdf6a6260148bb6c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d39f7e986b2e406189b91dff41d88bd5 (127.5.11.194:35497), 5c5e651a448342b39b026e3ad60c0373 (127.5.11.193:46537)
I20260808 22:30:03.272804 31629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:03.268366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:48813 (local address 127.5.11.193:46537)
0808 22:30:03.268525 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:03.268530 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:03.268581 (+ 51us) server_negotiation.cc:407] Connection header received
0808 22:30:03.268718 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:03.268721 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:03.268784 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:03.268870 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:30:03.269132 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:03.269885 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:03.271511 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:03.272190 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:03.272297 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:03.272441 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:03.272608 (+ 167us) server_negotiation.cc:299] Negotiation successful
0808 22:30:03.272677 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 22:30:03.272804 32064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:03.268210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:57321 (local address 127.5.11.194:35497)
0808 22:30:03.268326 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:03.268331 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:03.268407 (+ 76us) server_negotiation.cc:407] Connection header received
0808 22:30:03.268514 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:03.268517 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:03.268569 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:03.268653 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:30:03.268943 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:03.269762 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:03.271511 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:03.272135 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:03.272275 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:03.272328 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:03.272610 (+ 282us) server_negotiation.cc:299] Negotiation successful
0808 22:30:03.272677 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 22:30:03.272858 32046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:03.268126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:35497 (local address 127.5.11.195:57321)
0808 22:30:03.268301 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:03.268326 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:03.268422 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:03.268687 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:03.268692 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:03.268709 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:30:03.268814 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:03.268819 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:03.269872 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:03.269874 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:03.271386 (+ 1512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:03.271392 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:03.272275 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:03.272278 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:03.272448 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:03.272499 (+ 51us) client_negotiation.cc:769] Sending connection context
0808 22:30:03.272582 (+ 83us) client_negotiation.cc:240] Negotiation successful
0808 22:30:03.272703 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94,"mutex_wait_us":181}
I20260808 22:30:03.272873 32074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:03.268149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:46537 (local address 127.5.11.195:48813)
0808 22:30:03.268509 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:03.268523 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:03.268604 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:03.268904 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:03.268906 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:03.268926 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:30:03.269027 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:03.269031 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:03.269978 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:03.269981 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:03.271379 (+ 1398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:03.271392 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:03.272290 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:03.272293 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:03.272446 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:03.272503 (+ 57us) client_negotiation.cc:769] Sending connection context
0808 22:30:03.272582 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 22:30:03.272703 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":206,"thread_start_us":112,"threads_started":1}
I20260808 22:30:03.273530 31756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ececcc314a441eb04add447679bfd5" candidate_uuid: "a3751d7e254643ecbdf6a6260148bb6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d39f7e986b2e406189b91dff41d88bd5" is_pre_election: true
I20260808 22:30:03.273584 31552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ececcc314a441eb04add447679bfd5" candidate_uuid: "a3751d7e254643ecbdf6a6260148bb6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c5e651a448342b39b026e3ad60c0373" is_pre_election: true
I20260808 22:30:03.273867 31756 raft_consensus.cc:2454] T 97ececcc314a441eb04add447679bfd5 P d39f7e986b2e406189b91dff41d88bd5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3751d7e254643ecbdf6a6260148bb6c in term 0.
I20260808 22:30:03.273874 31552 raft_consensus.cc:2454] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a3751d7e254643ecbdf6a6260148bb6c in term 0.
I20260808 22:30:03.274328 31866 leader_election.cc:302] T 97ececcc314a441eb04add447679bfd5 P a3751d7e254643ecbdf6a6260148bb6c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c5e651a448342b39b026e3ad60c0373, a3751d7e254643ecbdf6a6260148bb6c; no voters:
I20260808 22:30:03.274623 32071 raft_consensus.cc:2781] T 97ececcc314a441eb04add447679bfd5 P a3751d7e254643ecbdf6a6260148bb6c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:30:03.274709 32071 raft_consensus.cc:482] T 97ececcc314a441eb04add447679bfd5 P a3751d7e254643ecbdf6a6260148bb6c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:30:03.274760 32071 raft_consensus.cc:3037] T 97ececcc314a441eb04add447679bfd5 P a3751d7e254643ecbdf6a6260148bb6c [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:30:03.275839 32071 raft_consensus.cc:504] T 97ececcc314a441eb04add447679bfd5 P a3751d7e254643ecbdf6a6260148bb6c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3751d7e254643ecbdf6a6260148bb6c" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37539 } } peers { permanent_uuid: "d39f7e986b2e406189b91dff41d88bd5" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35497 } } peers { permanent_uuid: "5c5e651a448342b39b026e3ad60c0373" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46537 } }
I20260808 22:30:03.276182 32071 leader_election.cc:288] T 97ececcc314a441eb04add447679bfd5 P a3751d7e254643ecbdf6a6260148bb6c [CANDIDATE]: Term 1 election: Requested vote from peers d39f7e986b2e406189b91dff41d88bd5 (127.5.11.194:35497), 5c5e651a448342b39b026e3ad60c0373 (127.5.11.193:46537)
I20260808 22:30:03.276539 31756 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ececcc314a441eb04add447679bfd5" candidate_uuid: "a3751d7e254643ecbdf6a6260148bb6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d39f7e986b2e406189b91dff41d88bd5"
I20260808 22:30:03.276576 31552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97ececcc314a441eb04add447679bfd5" candidate_uuid: "a3751d7e254643ecbdf6a6260148bb6c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c5e651a448342b39b026e3ad60c0373"
I20260808 22:30:03.276669 31756 raft_consensus.cc:3037] T 97ececcc314a441eb04add447679bfd5 P d39f7e986b2e406189b91dff41d88bd5 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:30:03.276684 31552 raft_consensus.cc:3037] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:30:03.277940 31756 raft_consensus.cc:2454] T 97ececcc314a441eb04add447679bfd5 P d39f7e986b2e406189b91dff41d88bd5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3751d7e254643ecbdf6a6260148bb6c in term 1.
I20260808 22:30:03.277940 31552 raft_consensus.cc:2454] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a3751d7e254643ecbdf6a6260148bb6c in term 1.
I20260808 22:30:03.278270 31868 leader_election.cc:302] T 97ececcc314a441eb04add447679bfd5 P a3751d7e254643ecbdf6a6260148bb6c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a3751d7e254643ecbdf6a6260148bb6c, d39f7e986b2e406189b91dff41d88bd5; no voters:
I20260808 22:30:03.278486 32071 raft_consensus.cc:2781] T 97ececcc314a441eb04add447679bfd5 P a3751d7e254643ecbdf6a6260148bb6c [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:30:03.278728 32071 raft_consensus.cc:686] T 97ececcc314a441eb04add447679bfd5 P a3751d7e254643ecbdf6a6260148bb6c [term 1 LEADER]: Becoming Leader. State: Replica: a3751d7e254643ecbdf6a6260148bb6c, State: Running, Role: LEADER
I20260808 22:30:03.279040 32071 consensus_queue.cc:237] T 97ececcc314a441eb04add447679bfd5 P a3751d7e254643ecbdf6a6260148bb6c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3751d7e254643ecbdf6a6260148bb6c" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37539 } } peers { permanent_uuid: "d39f7e986b2e406189b91dff41d88bd5" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35497 } } peers { permanent_uuid: "5c5e651a448342b39b026e3ad60c0373" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46537 } }
W20260808 22:30:03.281461 31618 tablet.cc:2369] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:30:03.282205 31335 catalog_manager.cc:5495] T 97ececcc314a441eb04add447679bfd5 P a3751d7e254643ecbdf6a6260148bb6c reported cstate change: term changed from 0 to 1, leader changed from <none> to a3751d7e254643ecbdf6a6260148bb6c (127.5.11.195). New cstate: current_term: 1 leader_uuid: "a3751d7e254643ecbdf6a6260148bb6c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3751d7e254643ecbdf6a6260148bb6c" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37539 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d39f7e986b2e406189b91dff41d88bd5" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c5e651a448342b39b026e3ad60c0373" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46537 } health_report { overall_health: UNKNOWN } } }
I20260808 22:30:03.295010 32059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:03.291237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43258 (local address 127.5.11.195:37539)
0808 22:30:03.291388 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:03.291394 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:03.291482 (+ 88us) server_negotiation.cc:407] Connection header received
0808 22:30:03.291668 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:03.291671 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:03.291724 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:03.291904 (+ 180us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:30:03.292147 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:03.292654 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:03.293519 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:03.293740 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:03.293879 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:03.294193 (+ 314us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:30:03.294584 (+ 391us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:30:03.294719 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:03.294788 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 22:30:03.294843 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260808 22:30:03.300882 31552 raft_consensus.cc:1261] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [term 1 FOLLOWER]: Refusing update from remote peer a3751d7e254643ecbdf6a6260148bb6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:30:03.300887 31756 raft_consensus.cc:1261] T 97ececcc314a441eb04add447679bfd5 P d39f7e986b2e406189b91dff41d88bd5 [term 1 FOLLOWER]: Refusing update from remote peer a3751d7e254643ecbdf6a6260148bb6c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:30:03.301617 32075 consensus_queue.cc:1035] T 97ececcc314a441eb04add447679bfd5 P a3751d7e254643ecbdf6a6260148bb6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "d39f7e986b2e406189b91dff41d88bd5" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:30:03.301869 32071 consensus_queue.cc:1035] T 97ececcc314a441eb04add447679bfd5 P a3751d7e254643ecbdf6a6260148bb6c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c5e651a448342b39b026e3ad60c0373" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:30:03.317230 31335 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:30:03.317410 31335 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33826:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:30:03.319731 31335 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8: Sending DeleteTablet for 3 replicas of tablet 97ececcc314a441eb04add447679bfd5
I20260808 22:30:03.320479 31729 tablet_service.cc:1514] Processing DeleteTablet for tablet 97ececcc314a441eb04add447679bfd5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:30:03 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:44858
I20260808 22:30:03.320477 31940 tablet_service.cc:1514] Processing DeleteTablet for tablet 97ececcc314a441eb04add447679bfd5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:30:03 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:43256
I20260808 22:30:03.320597 31522 tablet_service.cc:1514] Processing DeleteTablet for tablet 97ececcc314a441eb04add447679bfd5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:30:03 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:50828
I20260808 22:30:03.320783 31630 tablet_replica.cc:330] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373: stopping tablet replica
I20260808 22:30:03.320919 31630 raft_consensus.cc:2229] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:30:03.320955 32099 tablet_replica.cc:330] T 97ececcc314a441eb04add447679bfd5 P d39f7e986b2e406189b91dff41d88bd5: stopping tablet replica
I20260808 22:30:03.321045 31630 raft_consensus.cc:2258] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:30:03.321100 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 31395
I20260808 22:30:03.330106 32098 tablet_replica.cc:330] T 97ececcc314a441eb04add447679bfd5 P a3751d7e254643ecbdf6a6260148bb6c: stopping tablet replica
I20260808 22:30:03.330425 32098 raft_consensus.cc:2229] T 97ececcc314a441eb04add447679bfd5 P a3751d7e254643ecbdf6a6260148bb6c [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:30:03.330780 32098 raft_consensus.cc:2258] T 97ececcc314a441eb04add447679bfd5 P a3751d7e254643ecbdf6a6260148bb6c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:30:03.331085 32099 raft_consensus.cc:2229] T 97ececcc314a441eb04add447679bfd5 P d39f7e986b2e406189b91dff41d88bd5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:30:03.331423 32099 raft_consensus.cc:2258] T 97ececcc314a441eb04add447679bfd5 P d39f7e986b2e406189b91dff41d88bd5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:30:03.333019 32099 ts_tablet_manager.cc:1905] T 97ececcc314a441eb04add447679bfd5 P d39f7e986b2e406189b91dff41d88bd5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:30:03.333019 32098 ts_tablet_manager.cc:1905] T 97ececcc314a441eb04add447679bfd5 P a3751d7e254643ecbdf6a6260148bb6c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:30:03.335817 32099 ts_tablet_manager.cc:1918] T 97ececcc314a441eb04add447679bfd5 P d39f7e986b2e406189b91dff41d88bd5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:30:03.335817 32098 ts_tablet_manager.cc:1918] T 97ececcc314a441eb04add447679bfd5 P a3751d7e254643ecbdf6a6260148bb6c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:30:03.335922 32098 log.cc:1196] T 97ececcc314a441eb04add447679bfd5 P a3751d7e254643ecbdf6a6260148bb6c: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/wal/wals/97ececcc314a441eb04add447679bfd5
I20260808 22:30:03.336066 32099 log.cc:1196] T 97ececcc314a441eb04add447679bfd5 P d39f7e986b2e406189b91dff41d88bd5: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/wal/wals/97ececcc314a441eb04add447679bfd5
I20260808 22:30:03.336298 32098 ts_tablet_manager.cc:1939] T 97ececcc314a441eb04add447679bfd5 P a3751d7e254643ecbdf6a6260148bb6c: Deleting consensus metadata
I20260808 22:30:03.336576 32099 ts_tablet_manager.cc:1939] T 97ececcc314a441eb04add447679bfd5 P d39f7e986b2e406189b91dff41d88bd5: Deleting consensus metadata
W20260808 22:30:03.337190 31311 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.11.193:46537 (error 108)
I20260808 22:30:03.337556 31314 catalog_manager.cc:4841] TS a3751d7e254643ecbdf6a6260148bb6c (127.5.11.195:37539): tablet 97ececcc314a441eb04add447679bfd5 (table test-table [id=36444d5880134ccc976e6cd63d41bf3d]) successfully deleted
I20260808 22:30:03.338219 31313 catalog_manager.cc:4841] TS d39f7e986b2e406189b91dff41d88bd5 (127.5.11.194:35497): tablet 97ececcc314a441eb04add447679bfd5 (table test-table [id=36444d5880134ccc976e6cd63d41bf3d]) successfully deleted
I20260808 22:30:03.338413 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 31623
I20260808 22:30:03.342523 32063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:03.338704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:46537 (local address 127.0.0.1:50832)
0808 22:30:03.342382 (+ 3678us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:03.342448 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.193:46537: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3595}
W20260808 22:30:03.342720 31311 catalog_manager.cc:4584] TS 5c5e651a448342b39b026e3ad60c0373 (127.5.11.193:46537): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 97ececcc314a441eb04add447679bfd5: Network error: Client connection negotiation failed: client connection to 127.5.11.193:46537: connect: Connection refused (error 111)
I20260808 22:30:03.355058 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 31840
I20260808 22:30:03.368988 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 31288
I20260808 22:30:23.380800 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:40733
--webserver_interface=127.5.11.254
--webserver_port=36593
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:44513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:40733
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:30:23.496268 32105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:30:23.496500 32105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:30:23.496537 32105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:30:23.500275 32105 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260808 22:30:23.500316 32105 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260808 22:30:23.500331 32105 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260808 22:30:23.500344 32105 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:30:23.500397 32105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:30:23.500433 32105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:30:23.500463 32105 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:30:23.500494 32105 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:30:23.505496 32105 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:44513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:40733
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:40733
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=36593
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:30:23.506461 32105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:30:23.507459 32105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:30:23.516469 32112 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:23.516477 32113 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:23.516531 32105 server_base.cc:1034] running on GCE node
W20260808 22:30:23.516479 32115 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:23.517362 32105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:30:23.518507 32105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:30:23.519695 32105 hybrid_clock.cc:648] HybridClock initialized: now 1786228223519679 us; error 30 us; skew 500 ppm
Aug 08 22:30:23 dist-test-slave-7f01 krb5kdc[30435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228223, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:30:23.522504 32105 init.cc:377] Logged in from keytab as kudu/127.5.11.254@KRBTEST.COM (short username kudu)
I20260808 22:30:23.523366 32105 webserver.cc:460] Webserver started at http://127.5.11.254:36593/ using document root <none> and password file <none>
I20260808 22:30:23.524019 32105 fs_manager.cc:359] Metadata directory not provided
I20260808 22:30:23.524114 32105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:30:23.527872 32105 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 22:30:23.529989 32124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:23.531138 32105 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:30:23.531282 32105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "46ce689329e247e496c3f4fc2ddc42c8"
format_stamp: "Formatted at 2026-08-08 22:29:41 on dist-test-slave-7f01"
server_key: "59639cb1a4bf84aaab401c18376bc215"
server_key_iv: "69460f7363d9b53345ebba7c280c15d2"
server_key_version: "encryptionkey@0"
I20260808 22:30:23.531873 32105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:30:23.547328 32105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:30:23.550079 32105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:30:23.550277 32105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:30:23.557593 32105 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:40733
I20260808 22:30:23.557899 32199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:40733 every 8 connection(s)
I20260808 22:30:23.558688 32105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:30:23.559052 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 32105
I20260808 22:30:23.559561 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.193:46537
--local_ip_for_outbound_sockets=127.5.11.193
--tserver_master_addrs=127.5.11.254:40733
--webserver_port=33087
--webserver_interface=127.5.11.193
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:44513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:30:23.564049 32201 sys_catalog.cc:263] Verifying existing consensus state
I20260808 22:30:23.565761 32201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8: Bootstrap starting.
I20260808 22:30:23.583096 32201 log.cc:824] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8: Log is configured to *not* fsync() on all Append() calls
I20260808 22:30:23.594218 32201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 22:30:23.594609 32201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8: Bootstrap complete.
I20260808 22:30:23.598693 32201 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ce689329e247e496c3f4fc2ddc42c8" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 40733 } }
I20260808 22:30:23.599105 32201 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46ce689329e247e496c3f4fc2ddc42c8, State: Initialized, Role: FOLLOWER
I20260808 22:30:23.599608 32201 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ce689329e247e496c3f4fc2ddc42c8" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 40733 } }
I20260808 22:30:23.599776 32201 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:30:23.599871 32201 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:30:23.600003 32201 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [term 2 FOLLOWER]: Advancing to term 3
I20260808 22:30:23.601872 32201 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ce689329e247e496c3f4fc2ddc42c8" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 40733 } }
I20260808 22:30:23.602214 32201 leader_election.cc:302] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46ce689329e247e496c3f4fc2ddc42c8; no voters:
I20260808 22:30:23.602573 32201 leader_election.cc:288] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260808 22:30:23.602665 32209 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [term 3 FOLLOWER]: Leader election won for term 3
I20260808 22:30:23.602900 32209 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [term 3 LEADER]: Becoming Leader. State: Replica: 46ce689329e247e496c3f4fc2ddc42c8, State: Running, Role: LEADER
I20260808 22:30:23.603272 32209 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ce689329e247e496c3f4fc2ddc42c8" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 40733 } }
I20260808 22:30:23.603941 32201 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:30:23.604403 32212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "46ce689329e247e496c3f4fc2ddc42c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ce689329e247e496c3f4fc2ddc42c8" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 40733 } } }
I20260808 22:30:23.604527 32212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [sys.catalog]: This master's current role is: LEADER
I20260808 22:30:23.604768 32211 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46ce689329e247e496c3f4fc2ddc42c8. Latest consensus state: current_term: 3 leader_uuid: "46ce689329e247e496c3f4fc2ddc42c8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46ce689329e247e496c3f4fc2ddc42c8" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 40733 } } }
I20260808 22:30:23.604876 32211 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8 [sys.catalog]: This master's current role is: LEADER
I20260808 22:30:23.605183 32216 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:30:23.610493 32216 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:30:23.611251 32216 catalog_manager.cc:1259] Loaded cluster ID: 528252c11a89481295a51276d74ccbff
I20260808 22:30:23.611317 32216 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:30:23.613907 32216 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:30:23.614653 32216 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8: Loaded TSK: 0
I20260808 22:30:23.614739 32216 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 46ce689329e247e496c3f4fc2ddc42c8: Loaded TSK: 1
F20260808 22:30:23.615919 32216 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f97d79d99ad google::LogMessage::Fail() at ??:0
@ 0x7f97d79dc88a google::LogMessage::SendToLog() at ??:0
@ 0x7f97d79d94de google::LogMessage::Flush() at ??:0
@ 0x7f97d79daa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f97da69d18d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f97da695f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f97e233e7bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f97e231899c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f97e2319178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f97e235adf8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f97e237982b std::function<>::operator()() at ??:0
@ 0x7f97e2318b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f97e2319f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f97e2316579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f97e235a640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f97d9c2b3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f97d9c2bc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f97d9c2d3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f97d9c1c512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f97db2606db start_thread at ??:0
@ 0x7f97d64d761f clone at ??:0
W20260808 22:30:23.692999 32203 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:30:23.693230 32203 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:30:23.693272 32203 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:30:23.697556 32203 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:30:23.697604 32203 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:30:23.697661 32203 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.193
I20260808 22:30:23.703207 32203 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:44513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.193:46537
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.11.193
--webserver_port=33087
--tserver_master_addrs=127.5.11.254:40733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32203
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.193
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:30:23.704402 32203 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:30:23.705603 32203 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:30:23.716059 32265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:23.716127 32268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:23.716059 32266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:23.717607 32203 server_base.cc:1034] running on GCE node
I20260808 22:30:23.718116 32203 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:30:23.718770 32203 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:30:23.719921 32203 hybrid_clock.cc:648] HybridClock initialized: now 1786228223719891 us; error 47 us; skew 500 ppm
Aug 08 22:30:23 dist-test-slave-7f01 krb5kdc[30435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228223, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:30:23.723383 32203 init.cc:377] Logged in from keytab as kudu/127.5.11.193@KRBTEST.COM (short username kudu)
I20260808 22:30:23.724572 32203 webserver.cc:460] Webserver started at http://127.5.11.193:33087/ using document root <none> and password file <none>
I20260808 22:30:23.725322 32203 fs_manager.cc:359] Metadata directory not provided
I20260808 22:30:23.725419 32203 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:30:23.730619 32203 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 22:30:23.733386 32279 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:23.734712 32203 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260808 22:30:23.734854 32203 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/wal
uuid: "5c5e651a448342b39b026e3ad60c0373"
format_stamp: "Formatted at 2026-08-08 22:29:41 on dist-test-slave-7f01"
server_key: "fe6f540a9cf0f9cce39359d3e0e59f76"
server_key_iv: "03bae320e167e2d9d31f092ce6ec4fd1"
server_key_version: "encryptionkey@0"
I20260808 22:30:23.735487 32203 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:30:23.750654 32203 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:30:23.754029 32203 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:30:23.754242 32203 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:30:23.754855 32203 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:30:23.756059 32291 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260808 22:30:23.761587 32203 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260808 22:30:23.761663 32203 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260808 22:30:23.761759 32203 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260808 22:30:23.763434 32203 ts_tablet_manager.cc:610] Registered 1 tablets
I20260808 22:30:23.763506 32203 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260808 22:30:23.763863 32291 tablet_bootstrap.cc:492] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373: Bootstrap starting.
I20260808 22:30:23.774691 32203 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.193:46537
I20260808 22:30:23.774940 32445 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.193:46537 every 8 connection(s)
I20260808 22:30:23.775795 32203 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
I20260808 22:30:23.777560 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 32203
I20260808 22:30:23.778062 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.194:35497
--local_ip_for_outbound_sockets=127.5.11.194
--tserver_master_addrs=127.5.11.254:40733
--webserver_port=35203
--webserver_interface=127.5.11.194
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:44513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:30:23.786228 32291 log.cc:824] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373: Log is configured to *not* fsync() on all Append() calls
I20260808 22:30:23.796149 32291 tablet_bootstrap.cc:492] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260808 22:30:23.796578 32291 tablet_bootstrap.cc:492] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373: Bootstrap complete.
I20260808 22:30:23.798070 32291 ts_tablet_manager.cc:1397] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373: Time spent bootstrapping tablet: real 0.034s user 0.023s sys 0.008s
I20260808 22:30:23.801436 32291 raft_consensus.cc:348] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3751d7e254643ecbdf6a6260148bb6c" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37539 } } peers { permanent_uuid: "d39f7e986b2e406189b91dff41d88bd5" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35497 } } peers { permanent_uuid: "5c5e651a448342b39b026e3ad60c0373" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46537 } }
I20260808 22:30:23.801967 32291 raft_consensus.cc:729] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c5e651a448342b39b026e3ad60c0373, State: Initialized, Role: FOLLOWER
I20260808 22:30:23.802477 32291 consensus_queue.cc:260] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3751d7e254643ecbdf6a6260148bb6c" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37539 } } peers { permanent_uuid: "d39f7e986b2e406189b91dff41d88bd5" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35497 } } peers { permanent_uuid: "5c5e651a448342b39b026e3ad60c0373" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46537 } }
I20260808 22:30:23.803530 32291 ts_tablet_manager.cc:1428] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
W20260808 22:30:23.862210 32449 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:30:23.778005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.193:55349)
0808 22:30:23.781738 (+ 3733us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:23.781785 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:23.782585 (+ 800us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:23.861415 (+ 78830us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":3520,"thread_start_us":154,"threads_started":1}
W20260808 22:30:23.862874 32446 heartbeater.cc:644] Failed to heartbeat to 127.5.11.254:40733 (0 consecutive failures): Network error: Failed to ping master at 127.5.11.254:40733: Client connection negotiation failed: client connection to 127.5.11.254:40733: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260808 22:30:23.863657 32449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:23.863386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.193:43767)
0808 22:30:23.863519 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:23.863594 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260808 22:30:23.864415 32449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:23.864212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.193:37681)
0808 22:30:23.864319 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:23.864363 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 22:30:23.864645 32446 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260808 22:30:23.907275 32450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:30:23.907526 32450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:30:23.907604 32450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:30:23.911657 32450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:30:23.911731 32450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:30:23.911831 32450 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.194
I20260808 22:30:23.917388 32450 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:44513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.194:35497
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.11.194
--webserver_port=35203
--tserver_master_addrs=127.5.11.254:40733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32450
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.194
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:30:23.918370 32450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:30:23.919344 32450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:30:23.929390 32466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:23.929399 32467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:23.929399 32469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:23.929628 32450 server_base.cc:1034] running on GCE node
I20260808 22:30:23.930068 32450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:30:23.930692 32450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:30:23.931878 32450 hybrid_clock.cc:648] HybridClock initialized: now 1786228223931843 us; error 45 us; skew 500 ppm
Aug 08 22:30:23 dist-test-slave-7f01 krb5kdc[30435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228223, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:30:23.934612 32450 init.cc:377] Logged in from keytab as kudu/127.5.11.194@KRBTEST.COM (short username kudu)
I20260808 22:30:23.935590 32450 webserver.cc:460] Webserver started at http://127.5.11.194:35203/ using document root <none> and password file <none>
I20260808 22:30:23.936213 32450 fs_manager.cc:359] Metadata directory not provided
I20260808 22:30:23.936306 32450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:30:23.940269 32450 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:30:23.942235 32478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:23.943423 32450 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:30:23.943573 32450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/wal
uuid: "d39f7e986b2e406189b91dff41d88bd5"
format_stamp: "Formatted at 2026-08-08 22:29:41 on dist-test-slave-7f01"
server_key: "3c418455c336e53ad4d1b191e1aadf40"
server_key_iv: "822c48bc029124f31c9305113c897a11"
server_key_version: "encryptionkey@0"
I20260808 22:30:23.944190 32450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:30:23.962745 32450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:30:23.966293 32450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:30:23.966513 32450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:30:23.967145 32450 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:30:23.968055 32450 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:30:23.968125 32450 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:23.968192 32450 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:30:23.968240 32450 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:23.977279 32450 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.194:35497
I20260808 22:30:23.977298 32642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.194:35497 every 8 connection(s)
I20260808 22:30:23.978332 32450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
I20260808 22:30:23.982299 32645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:23.980505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.194:43533)
0808 22:30:23.980948 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:23.981057 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":111,"threads_started":1}
W20260808 22:30:23.982827 32643 heartbeater.cc:644] Failed to heartbeat to 127.5.11.254:40733 (0 consecutive failures): Network error: Failed to ping master at 127.5.11.254:40733: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
I20260808 22:30:23.983417 32645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:23.983208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.194:34093)
0808 22:30:23.983315 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:23.983360 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 22:30:23.984047 32645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:23.983868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.194:45727)
0808 22:30:23.983960 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:23.984001 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 22:30:23.984230 32643 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 22:30:23.987931 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 32450
I20260808 22:30:23.988369 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.195:37539
--local_ip_for_outbound_sockets=127.5.11.195
--tserver_master_addrs=127.5.11.254:40733
--webserver_port=36883
--webserver_interface=127.5.11.195
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:44513
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:30:24.102602 32647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:30:24.102854 32647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:30:24.102895 32647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:30:24.106740 32647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:30:24.106813 32647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:30:24.106870 32647 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.195
I20260808 22:30:24.111862 32647 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:44513
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.195:37539
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.11.195
--webserver_port=36883
--tserver_master_addrs=127.5.11.254:40733
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32647
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.195
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:30:24.112941 32647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:30:24.113997 32647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:30:24.123450 32656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:24.123517 32647 server_base.cc:1034] running on GCE node
W20260808 22:30:24.123440 32660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:24.123440 32658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:24.124174 32647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:30:24.124780 32647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:30:24.125982 32647 hybrid_clock.cc:648] HybridClock initialized: now 1786228224125933 us; error 55 us; skew 500 ppm
Aug 08 22:30:24 dist-test-slave-7f01 krb5kdc[30435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228224, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:30:24.128715 32647 init.cc:377] Logged in from keytab as kudu/127.5.11.195@KRBTEST.COM (short username kudu)
I20260808 22:30:24.129554 32647 webserver.cc:460] Webserver started at http://127.5.11.195:36883/ using document root <none> and password file <none>
I20260808 22:30:24.130117 32647 fs_manager.cc:359] Metadata directory not provided
I20260808 22:30:24.130211 32647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:30:24.133932 32647 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260808 22:30:24.135988 32670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:24.137190 32647 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:30:24.137328 32647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/wal
uuid: "a3751d7e254643ecbdf6a6260148bb6c"
format_stamp: "Formatted at 2026-08-08 22:29:42 on dist-test-slave-7f01"
server_key: "8b6aca52aa2e7fb7e7cffd43774121c3"
server_key_iv: "f9a286397a776843f2ae4247923ad1fe"
server_key_version: "encryptionkey@0"
I20260808 22:30:24.137861 32647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:30:24.154816 32647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:30:24.157567 32647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:30:24.157766 32647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:30:24.158373 32647 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:30:24.159197 32647 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:30:24.159276 32647 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:24.159371 32647 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:30:24.159425 32647 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:24.168486 32647 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.195:37539
I20260808 22:30:24.168524 372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.195:37539 every 8 connection(s)
I20260808 22:30:24.169459 32647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
I20260808 22:30:24.173301 375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:24.171542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.195:37075)
0808 22:30:24.172515 (+ 973us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:24.172624 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":744,"thread_start_us":606,"threads_started":1}
W20260808 22:30:24.173823 373 heartbeater.cc:644] Failed to heartbeat to 127.5.11.254:40733 (0 consecutive failures): Network error: Failed to ping master at 127.5.11.254:40733: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
I20260808 22:30:24.174387 375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:24.174189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.195:44165)
0808 22:30:24.174291 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:24.174335 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260808 22:30:24.174954 375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:24.174779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.195:47293)
0808 22:30:24.174868 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:24.174907 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260808 22:30:24.175127 373 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260808 22:30:24.176015 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 32647
I20260808 22:30:24.865720 378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:24.865256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.193:51495)
0808 22:30:24.865538 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:24.865599 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":78,"threads_started":1}
I20260808 22:30:24.978253 379 raft_consensus.cc:482] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:30:24.978492 379 raft_consensus.cc:504] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3751d7e254643ecbdf6a6260148bb6c" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37539 } } peers { permanent_uuid: "d39f7e986b2e406189b91dff41d88bd5" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35497 } } peers { permanent_uuid: "5c5e651a448342b39b026e3ad60c0373" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46537 } }
I20260808 22:30:24.979665 379 leader_election.cc:288] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3751d7e254643ecbdf6a6260148bb6c (127.5.11.195:37539), d39f7e986b2e406189b91dff41d88bd5 (127.5.11.194:35497)
I20260808 22:30:24.985208 388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:24.984776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.194:53811)
0808 22:30:24.985062 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:24.985112 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":87,"threads_started":1}
Aug 08 22:30:24 dist-test-slave-7f01 krb5kdc[30435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228223, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for kudu/127.5.11.194@KRBTEST.COM
Aug 08 22:30:24 dist-test-slave-7f01 krb5kdc[30435](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228223, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for kudu/127.5.11.195@KRBTEST.COM
I20260808 22:30:24.994282 380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:24.979935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:35497 (local address 127.5.11.193:40535)
0808 22:30:24.980468 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:24.980482 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:24.980574 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:24.982125 (+ 1551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:24.982128 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:24.982625 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:30:24.983098 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:24.983112 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:24.984494 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:24.984497 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:24.985950 (+ 1453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:24.985962 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:24.986664 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:24.986668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:24.986878 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:24.987535 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:30:24.987552 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:30:24.989251 (+ 1699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:30:24.991504 (+ 2253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:30:24.991510 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:30:24.991525 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:30:24.991967 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:30:24.992339 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:30:24.992342 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:30:24.992344 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:30:24.992539 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:30:24.993302 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:30:24.993311 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:30:24.993550 (+ 239us) client_negotiation.cc:769] Sending connection context
0808 22:30:24.993725 (+ 175us) client_negotiation.cc:240] Negotiation successful
0808 22:30:24.994072 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":472,"mutex_wait_us":178,"spinlock_wait_cycles":27648,"thread_start_us":72,"threads_started":1}
I20260808 22:30:24.994294 382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:24.980216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:51247 (local address 127.5.11.195:37539)
0808 22:30:24.980692 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:24.980704 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:24.980739 (+ 35us) server_negotiation.cc:407] Connection header received
0808 22:30:24.981359 (+ 620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:24.981364 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:24.981693 (+ 329us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:24.982007 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:24.983283 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:24.984330 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:24.986849 (+ 2519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:24.987411 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:24.987580 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:24.989799 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:24.989803 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:24.989813 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:30:24.989840 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:24.991706 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:24.992224 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:24.992230 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:24.992237 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:24.992412 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:24.992767 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:24.992770 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:24.992772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:24.993217 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:24.993368 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:24.993846 (+ 478us) server_negotiation.cc:299] Negotiation successful
0808 22:30:24.994105 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":101,"threads_started":1}
I20260808 22:30:24.994341 378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:24.979884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:37539 (local address 127.5.11.193:51247)
0808 22:30:24.980063 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:24.980080 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:24.980188 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:24.982053 (+ 1865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:24.982059 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:24.982615 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:30:24.983098 (+ 483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:24.983112 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:24.984477 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:24.984481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:24.985950 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:24.985962 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:24.987536 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:24.987540 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:24.987623 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:24.988132 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:30:24.988144 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:30:24.989621 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:30:24.991850 (+ 2229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:30:24.991853 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:30:24.991855 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:30:24.992101 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:30:24.992553 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:30:24.992556 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:30:24.992558 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:30:24.992658 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:30:24.993362 (+ 704us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:30:24.993365 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:30:24.993544 (+ 179us) client_negotiation.cc:769] Sending connection context
0808 22:30:24.993726 (+ 182us) client_negotiation.cc:240] Negotiation successful
0808 22:30:24.994159 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":408,"spinlock_wait_cycles":9088}
I20260808 22:30:24.994333 383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:24.980306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:40535 (local address 127.5.11.194:35497)
0808 22:30:24.980692 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:24.980705 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:24.980742 (+ 37us) server_negotiation.cc:407] Connection header received
0808 22:30:24.981524 (+ 782us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:24.981532 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:24.981890 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:24.982166 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:24.983276 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:24.984319 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:24.986081 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:24.986543 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:24.986664 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:24.989515 (+ 2851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:24.989524 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:24.989542 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:30:24.989579 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:24.991374 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:24.992094 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:24.992100 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:24.992106 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:24.992262 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:24.992636 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:24.992640 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:24.992641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:24.993180 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:24.993285 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:24.993866 (+ 581us) server_negotiation.cc:299] Negotiation successful
0808 22:30:24.994149 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":82,"threads_started":1}
W20260808 22:30:24.996953 32286 leader_election.cc:341] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a3751d7e254643ecbdf6a6260148bb6c (127.5.11.195:37539): Not found: Tablet not found: 97ececcc314a441eb04add447679bfd5
W20260808 22:30:24.997110 32284 leader_election.cc:341] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d39f7e986b2e406189b91dff41d88bd5 (127.5.11.194:35497): Not found: Tablet not found: 97ececcc314a441eb04add447679bfd5
I20260808 22:30:24.997184 32284 leader_election.cc:302] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c5e651a448342b39b026e3ad60c0373; no voters: a3751d7e254643ecbdf6a6260148bb6c, d39f7e986b2e406189b91dff41d88bd5
I20260808 22:30:24.997483 379 raft_consensus.cc:2726] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 22:30:25.176031 389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:25.175603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.195:33479)
0808 22:30:25.175884 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:25.175940 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":83,"threads_started":1}
I20260808 22:30:25.866889 390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:25.866472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.193:42537)
0808 22:30:25.866717 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:25.866785 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":80,"threads_started":1}
I20260808 22:30:25.986487 392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:25.986100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.194:51299)
0808 22:30:25.986341 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:25.986392 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":74,"threads_started":1}
I20260808 22:30:26.177152 394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:26.176725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.195:59717)
0808 22:30:26.177000 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:26.177052 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":122,"threads_started":1}
I20260808 22:30:26.574831 395 raft_consensus.cc:482] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:30:26.574980 395 raft_consensus.cc:504] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3751d7e254643ecbdf6a6260148bb6c" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37539 } } peers { permanent_uuid: "d39f7e986b2e406189b91dff41d88bd5" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35497 } } peers { permanent_uuid: "5c5e651a448342b39b026e3ad60c0373" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46537 } }
I20260808 22:30:26.575340 395 leader_election.cc:288] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3751d7e254643ecbdf6a6260148bb6c (127.5.11.195:37539), d39f7e986b2e406189b91dff41d88bd5 (127.5.11.194:35497)
W20260808 22:30:26.576272 32286 leader_election.cc:341] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a3751d7e254643ecbdf6a6260148bb6c (127.5.11.195:37539): Not found: Tablet not found: 97ececcc314a441eb04add447679bfd5
W20260808 22:30:26.576383 32284 leader_election.cc:341] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d39f7e986b2e406189b91dff41d88bd5 (127.5.11.194:35497): Not found: Tablet not found: 97ececcc314a441eb04add447679bfd5
I20260808 22:30:26.576447 32284 leader_election.cc:302] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c5e651a448342b39b026e3ad60c0373; no voters: a3751d7e254643ecbdf6a6260148bb6c, d39f7e986b2e406189b91dff41d88bd5
I20260808 22:30:26.576582 395 raft_consensus.cc:2726] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 22:30:26.868228 396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:26.867727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.193:36271)
0808 22:30:26.868067 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:26.868124 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":85,"threads_started":1}
I20260808 22:30:26.987841 397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:26.987328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.194:58409)
0808 22:30:26.987610 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:26.987670 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":124,"threads_started":1}
I20260808 22:30:27.178253 398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:27.177810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.195:36147)
0808 22:30:27.178055 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:27.178115 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":75,"threads_started":1}
I20260808 22:30:27.869916 400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:27.869288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.193:41885)
0808 22:30:27.869734 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:27.869807 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":120,"threads_started":1}
I20260808 22:30:27.989241 401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:27.988751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.194:45525)
0808 22:30:27.989056 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:27.989128 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":84,"threads_started":1}
I20260808 22:30:28.179632 402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:28.179140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.195:55639)
0808 22:30:28.179471 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:28.179538 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":100,"threads_started":1}
I20260808 22:30:28.206669 403 raft_consensus.cc:482] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:30:28.206836 403 raft_consensus.cc:504] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3751d7e254643ecbdf6a6260148bb6c" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37539 } } peers { permanent_uuid: "d39f7e986b2e406189b91dff41d88bd5" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35497 } } peers { permanent_uuid: "5c5e651a448342b39b026e3ad60c0373" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46537 } }
I20260808 22:30:28.207389 403 leader_election.cc:288] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3751d7e254643ecbdf6a6260148bb6c (127.5.11.195:37539), d39f7e986b2e406189b91dff41d88bd5 (127.5.11.194:35497)
W20260808 22:30:28.208225 32286 leader_election.cc:341] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a3751d7e254643ecbdf6a6260148bb6c (127.5.11.195:37539): Not found: Tablet not found: 97ececcc314a441eb04add447679bfd5
W20260808 22:30:28.208320 32284 leader_election.cc:341] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d39f7e986b2e406189b91dff41d88bd5 (127.5.11.194:35497): Not found: Tablet not found: 97ececcc314a441eb04add447679bfd5
I20260808 22:30:28.208448 32284 leader_election.cc:302] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c5e651a448342b39b026e3ad60c0373; no voters: a3751d7e254643ecbdf6a6260148bb6c, d39f7e986b2e406189b91dff41d88bd5
I20260808 22:30:28.208650 403 raft_consensus.cc:2726] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 22:30:28.871464 404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:28.870885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.193:38101)
0808 22:30:28.871255 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:28.871320 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":99,"threads_started":1}
I20260808 22:30:28.990729 405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:28.990204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.194:44093)
0808 22:30:28.990531 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:28.990592 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":84,"threads_started":1}
I20260808 22:30:29.180850 408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:29.180442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.195:54003)
0808 22:30:29.180701 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:29.180757 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":82,"threads_started":1}
I20260808 22:30:29.872862 409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:29.872403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.193:42109)
0808 22:30:29.872694 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:29.872752 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":100,"threads_started":1}
I20260808 22:30:29.992137 411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:29.991587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.194:50067)
0808 22:30:29.991908 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:29.991996 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":81,"threads_started":1}
I20260808 22:30:30.181983 412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:30.181569 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.195:36051)
0808 22:30:30.181812 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:30.181866 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":78,"threads_started":1}
I20260808 22:30:30.874162 414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:30.873713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.193:36351)
0808 22:30:30.873993 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:30.874052 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":88,"threads_started":1}
I20260808 22:30:30.993418 415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:30.992926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.194:47921)
0808 22:30:30.993266 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:30.993319 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":116,"threads_started":1}
I20260808 22:30:31.183188 416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:31.182703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.195:57779)
0808 22:30:31.183045 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:31.183100 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":104,"threads_started":1}
I20260808 22:30:31.875428 419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:31.875003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.193:45711)
0808 22:30:31.875267 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:31.875324 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":94,"threads_started":1}
I20260808 22:30:31.994652 420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:31.994220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.194:51783)
0808 22:30:31.994492 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:31.994551 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":84,"threads_started":1}
I20260808 22:30:32.184381 421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:32.183977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.195:34753)
0808 22:30:32.184229 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:32.184286 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":89,"threads_started":1}
I20260808 22:30:32.876703 422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:32.876263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.193:53603)
0808 22:30:32.876532 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:32.876585 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":81,"threads_started":1}
I20260808 22:30:32.995954 423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:32.995469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.194:41157)
0808 22:30:32.995781 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:32.995844 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":81,"threads_started":1}
I20260808 22:30:33.185544 425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:33.185110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.195:59279)
0808 22:30:33.185376 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:33.185444 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":75,"threads_started":1}
I20260808 22:30:33.240123 426 raft_consensus.cc:482] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:30:33.240271 426 raft_consensus.cc:504] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3751d7e254643ecbdf6a6260148bb6c" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37539 } } peers { permanent_uuid: "d39f7e986b2e406189b91dff41d88bd5" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35497 } } peers { permanent_uuid: "5c5e651a448342b39b026e3ad60c0373" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46537 } }
I20260808 22:30:33.240670 426 leader_election.cc:288] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3751d7e254643ecbdf6a6260148bb6c (127.5.11.195:37539), d39f7e986b2e406189b91dff41d88bd5 (127.5.11.194:35497)
W20260808 22:30:33.241511 32284 leader_election.cc:341] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d39f7e986b2e406189b91dff41d88bd5 (127.5.11.194:35497): Not found: Tablet not found: 97ececcc314a441eb04add447679bfd5
W20260808 22:30:33.241636 32286 leader_election.cc:341] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a3751d7e254643ecbdf6a6260148bb6c (127.5.11.195:37539): Not found: Tablet not found: 97ececcc314a441eb04add447679bfd5
I20260808 22:30:33.241835 32286 leader_election.cc:302] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c5e651a448342b39b026e3ad60c0373; no voters: a3751d7e254643ecbdf6a6260148bb6c, d39f7e986b2e406189b91dff41d88bd5
I20260808 22:30:33.242008 426 raft_consensus.cc:2726] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 22:30:33.878154 427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:33.877723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.193:44093)
0808 22:30:33.877995 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:33.878048 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":95,"threads_started":1}
I20260808 22:30:33.997105 429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:33.996677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.194:59423)
0808 22:30:33.996941 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:33.997003 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":100,"threads_started":1}
I20260808 22:30:34.186681 430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:34.186280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.195:43065)
0808 22:30:34.186531 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:34.186584 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":92,"threads_started":1}
I20260808 22:30:34.879423 431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:34.879008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.193:53749)
0808 22:30:34.879264 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:34.879320 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":81,"threads_started":1}
I20260808 22:30:34.998450 432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:34.997988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.194:39505)
0808 22:30:34.998253 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:34.998333 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":99,"threads_started":1}
I20260808 22:30:35.117805 433 raft_consensus.cc:482] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:30:35.117951 433 raft_consensus.cc:504] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3751d7e254643ecbdf6a6260148bb6c" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37539 } } peers { permanent_uuid: "d39f7e986b2e406189b91dff41d88bd5" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35497 } } peers { permanent_uuid: "5c5e651a448342b39b026e3ad60c0373" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46537 } }
I20260808 22:30:35.118458 433 leader_election.cc:288] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3751d7e254643ecbdf6a6260148bb6c (127.5.11.195:37539), d39f7e986b2e406189b91dff41d88bd5 (127.5.11.194:35497)
W20260808 22:30:35.119230 32286 leader_election.cc:341] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a3751d7e254643ecbdf6a6260148bb6c (127.5.11.195:37539): Not found: Tablet not found: 97ececcc314a441eb04add447679bfd5
W20260808 22:30:35.119445 32284 leader_election.cc:341] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d39f7e986b2e406189b91dff41d88bd5 (127.5.11.194:35497): Not found: Tablet not found: 97ececcc314a441eb04add447679bfd5
I20260808 22:30:35.119549 32284 leader_election.cc:302] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c5e651a448342b39b026e3ad60c0373; no voters: a3751d7e254643ecbdf6a6260148bb6c, d39f7e986b2e406189b91dff41d88bd5
I20260808 22:30:35.119729 433 raft_consensus.cc:2726] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 22:30:35.188092 434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:35.187466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.195:56521)
0808 22:30:35.187919 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:35.187991 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":83,"threads_started":1}
I20260808 22:30:35.880831 435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:35.880295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.193:37807)
0808 22:30:35.880616 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:35.880700 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":100,"threads_started":1}
I20260808 22:30:35.999696 436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:35.999152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.194:54355)
0808 22:30:35.999534 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:35.999593 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":96,"threads_started":1}
I20260808 22:30:36.189285 437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:36.188852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.195:52201)
0808 22:30:36.189121 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:36.189180 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":90,"threads_started":1}
I20260808 22:30:36.881935 438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:36.881545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.193:52001)
0808 22:30:36.881780 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:36.881834 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":78,"threads_started":1}
I20260808 22:30:37.000957 439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:37.000541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.194:54167)
0808 22:30:37.000793 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:37.000851 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":76,"threads_started":1}
I20260808 22:30:37.190625 440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:37.190179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.195:37749)
0808 22:30:37.190470 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:37.190525 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":109,"threads_started":1}
I20260808 22:30:37.883245 442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:37.882767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.193:51235)
0808 22:30:37.883089 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:37.883143 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":150,"threads_started":1}
I20260808 22:30:38.002115 444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:38.001707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.194:60877)
0808 22:30:38.001959 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:38.002012 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":86,"threads_started":1}
I20260808 22:30:38.191954 446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:38.191411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.195:35911)
0808 22:30:38.191715 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:38.191830 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":80,"threads_started":1}
I20260808 22:30:38.884913 447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:38.884381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.193:38559)
0808 22:30:38.884710 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:38.884780 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":89,"threads_started":1}
I20260808 22:30:38.928046 449 raft_consensus.cc:482] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:30:38.928237 449 raft_consensus.cc:504] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a3751d7e254643ecbdf6a6260148bb6c" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 37539 } } peers { permanent_uuid: "d39f7e986b2e406189b91dff41d88bd5" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 35497 } } peers { permanent_uuid: "5c5e651a448342b39b026e3ad60c0373" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 46537 } }
I20260808 22:30:38.928710 449 leader_election.cc:288] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a3751d7e254643ecbdf6a6260148bb6c (127.5.11.195:37539), d39f7e986b2e406189b91dff41d88bd5 (127.5.11.194:35497)
W20260808 22:30:38.929616 32284 leader_election.cc:341] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer d39f7e986b2e406189b91dff41d88bd5 (127.5.11.194:35497): Not found: Tablet not found: 97ececcc314a441eb04add447679bfd5
W20260808 22:30:38.929749 32286 leader_election.cc:341] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer a3751d7e254643ecbdf6a6260148bb6c (127.5.11.195:37539): Not found: Tablet not found: 97ececcc314a441eb04add447679bfd5
I20260808 22:30:38.929847 32286 leader_election.cc:302] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5c5e651a448342b39b026e3ad60c0373; no voters: a3751d7e254643ecbdf6a6260148bb6c, d39f7e986b2e406189b91dff41d88bd5
I20260808 22:30:38.930047 449 raft_consensus.cc:2726] T 97ececcc314a441eb04add447679bfd5 P 5c5e651a448342b39b026e3ad60c0373 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260808 22:30:39.003661 450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:39.003167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.194:42371)
0808 22:30:39.003491 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:39.003553 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":107,"threads_started":1}
I20260808 22:30:39.188795 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 32203
I20260808 22:30:39.197583 452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:39.194965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:40733 (local address 127.5.11.195:52729)
0808 22:30:39.197362 (+ 2397us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:39.197446 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.254:40733: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2271,"thread_start_us":427,"threads_started":1}
I20260808 22:30:39.205638 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 32450
I20260808 22:30:39.216125 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 32647
2026-08-08T22:30:39Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59609 ms)
[----------] 30 tests from SecurityITest (247404 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:30:39 dist-test-slave-7f01 krb5kdc[455](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:30:39 dist-test-slave-7f01 krb5kdc[455](info): set up 2 sockets
Aug 08 22:30:39 dist-test-slave-7f01 krb5kdc[455](info): commencing operation
krb5kdc: starting...
W20260808 22:30:40.664636 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:30:40 dist-test-slave-7f01 krb5kdc[455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:30:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:30:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.11.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
I20260808 22:30:40.794906 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:34267
--webserver_interface=127.5.11.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:44607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:34267
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:30:40.929770 474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:30:40.930059 474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:30:40.930128 474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:30:40.934028 474 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:30:40.934113 474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:30:40.934152 474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:30:40.934204 474 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:30:40.934245 474 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:30:40.939327 474 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:44607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:34267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:34267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:30:40.940371 474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:30:40.941457 474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:30:40.950248 485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:40.950259 488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:40.950260 484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:40.950603 474 server_base.cc:1034] running on GCE node
I20260808 22:30:40.951201 474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:30:40.952371 474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:30:40.953565 474 hybrid_clock.cc:648] HybridClock initialized: now 1786228240953529 us; error 44 us; skew 500 ppm
Aug 08 22:30:40 dist-test-slave-7f01 krb5kdc[455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228240, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:30:40.956405 474 init.cc:377] Logged in from keytab as kudu/127.5.11.254@KRBTEST.COM (short username kudu)
I20260808 22:30:40.957316 474 webserver.cc:460] Webserver started at http://127.5.11.254:41321/ using document root <none> and password file <none>
I20260808 22:30:40.957893 474 fs_manager.cc:359] Metadata directory not provided
I20260808 22:30:40.957980 474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:30:40.958212 474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:30:40.960361 474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "c8d903cbe00f4b35a288bb5a673ff548"
format_stamp: "Formatted at 2026-08-08 22:30:40 on dist-test-slave-7f01"
server_key: "8baba65ae786662cf0cbaa72a9603fe7"
server_key_iv: "73762a334e3665bd588eef1d583ea082"
server_key_version: "encryptionkey@0"
I20260808 22:30:40.960934 474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "c8d903cbe00f4b35a288bb5a673ff548"
format_stamp: "Formatted at 2026-08-08 22:30:40 on dist-test-slave-7f01"
server_key: "8baba65ae786662cf0cbaa72a9603fe7"
server_key_iv: "73762a334e3665bd588eef1d583ea082"
server_key_version: "encryptionkey@0"
I20260808 22:30:40.964813 474 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260808 22:30:40.967222 503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:40.968418 474 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 22:30:40.968562 474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "c8d903cbe00f4b35a288bb5a673ff548"
format_stamp: "Formatted at 2026-08-08 22:30:40 on dist-test-slave-7f01"
server_key: "8baba65ae786662cf0cbaa72a9603fe7"
server_key_iv: "73762a334e3665bd588eef1d583ea082"
server_key_version: "encryptionkey@0"
I20260808 22:30:40.968670 474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:30:40.999213 474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:30:41.002215 474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:30:41.002441 474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:30:41.009802 474 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:34267
I20260808 22:30:41.009840 581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:34267 every 8 connection(s)
I20260808 22:30:41.010747 474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:30:41.013496 583 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:30:41.015698 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 474
I20260808 22:30:41.015879 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:30:41.016183 5167 external_mini_cluster.cc:1428] Setting key a1818c70cdac4c06dae18058834a15cd
I20260808 22:30:41.019394 583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8d903cbe00f4b35a288bb5a673ff548: Bootstrap starting.
I20260808 22:30:41.021662 583 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8d903cbe00f4b35a288bb5a673ff548: Neither blocks nor log segments found. Creating new log.
I20260808 22:30:41.022466 583 log.cc:824] T 00000000000000000000000000000000 P c8d903cbe00f4b35a288bb5a673ff548: Log is configured to *not* fsync() on all Append() calls
I20260808 22:30:41.024423 583 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8d903cbe00f4b35a288bb5a673ff548: No bootstrap required, opened a new log
Aug 08 22:30:41 dist-test-slave-7f01 krb5kdc[455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228240, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:30:41.027877 583 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8d903cbe00f4b35a288bb5a673ff548 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8d903cbe00f4b35a288bb5a673ff548" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 34267 } }
I20260808 22:30:41.028097 583 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8d903cbe00f4b35a288bb5a673ff548 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:30:41.028172 583 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8d903cbe00f4b35a288bb5a673ff548 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8d903cbe00f4b35a288bb5a673ff548, State: Initialized, Role: FOLLOWER
I20260808 22:30:41.028631 583 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8d903cbe00f4b35a288bb5a673ff548 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8d903cbe00f4b35a288bb5a673ff548" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 34267 } }
I20260808 22:30:41.028785 583 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8d903cbe00f4b35a288bb5a673ff548 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:30:41.028877 583 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8d903cbe00f4b35a288bb5a673ff548 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:30:41.028995 583 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8d903cbe00f4b35a288bb5a673ff548 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:30:41.029853 583 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8d903cbe00f4b35a288bb5a673ff548 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8d903cbe00f4b35a288bb5a673ff548" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 34267 } }
I20260808 22:30:41.030225 583 leader_election.cc:302] T 00000000000000000000000000000000 P c8d903cbe00f4b35a288bb5a673ff548 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8d903cbe00f4b35a288bb5a673ff548; no voters:
I20260808 22:30:41.030537 583 leader_election.cc:288] T 00000000000000000000000000000000 P c8d903cbe00f4b35a288bb5a673ff548 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:30:41.030639 591 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8d903cbe00f4b35a288bb5a673ff548 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:30:41.030850 591 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8d903cbe00f4b35a288bb5a673ff548 [term 1 LEADER]: Becoming Leader. State: Replica: c8d903cbe00f4b35a288bb5a673ff548, State: Running, Role: LEADER
I20260808 22:30:41.031018 587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:41.017575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36692 (local address 127.5.11.254:34267)
0808 22:30:41.018000 (+ 425us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:41.018016 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:41.018051 (+ 35us) server_negotiation.cc:407] Connection header received
0808 22:30:41.018688 (+ 637us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:41.018705 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:41.019026 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:41.019371 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:41.020181 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:41.021188 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:41.022112 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:41.022629 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:41.022800 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:41.025473 (+ 2673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:41.025479 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:41.025491 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:30:41.025517 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:41.027850 (+ 2333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:41.028354 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:41.028359 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:41.028363 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:41.028476 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:41.028761 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:41.028765 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:41.028767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:41.029290 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:41.029517 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:41.029766 (+ 249us) server_negotiation.cc:299] Negotiation successful
0808 22:30:41.030083 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":135,"threads_started":1}
I20260808 22:30:41.031149 591 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8d903cbe00f4b35a288bb5a673ff548 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8d903cbe00f4b35a288bb5a673ff548" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 34267 } }
I20260808 22:30:41.031821 583 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8d903cbe00f4b35a288bb5a673ff548 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:30:41.032338 593 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8d903cbe00f4b35a288bb5a673ff548 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8d903cbe00f4b35a288bb5a673ff548. Latest consensus state: current_term: 1 leader_uuid: "c8d903cbe00f4b35a288bb5a673ff548" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8d903cbe00f4b35a288bb5a673ff548" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 34267 } } }
I20260808 22:30:41.032467 593 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8d903cbe00f4b35a288bb5a673ff548 [sys.catalog]: This master's current role is: LEADER
I20260808 22:30:41.033263 596 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:30:41.033798 592 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8d903cbe00f4b35a288bb5a673ff548 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c8d903cbe00f4b35a288bb5a673ff548" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8d903cbe00f4b35a288bb5a673ff548" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 34267 } } }
I20260808 22:30:41.033934 592 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8d903cbe00f4b35a288bb5a673ff548 [sys.catalog]: This master's current role is: LEADER
I20260808 22:30:41.036494 596 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:30:41.042321 596 catalog_manager.cc:1347] Generated new cluster ID: 5859adff3fbf4aa6bc23dbe4179d6de7
I20260808 22:30:41.042394 596 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:30:41.051602 596 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:30:41.052809 596 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:30:41.060838 596 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c8d903cbe00f4b35a288bb5a673ff548: Generated new TSK 0
I20260808 22:30:41.061599 596 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.11.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab.
Entry for principal kudu/127.5.11.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab.
I20260808 22:30:41.098604 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.193:0
--local_ip_for_outbound_sockets=127.5.11.193
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:34267
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:44607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:30:41.212213 622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:30:41.212478 622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:30:41.212551 622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:30:41.216475 622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:30:41.216552 622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:30:41.216639 622 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.193
I20260808 22:30:41.221639 622 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:44607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:34267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.193
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:30:41.222658 622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:30:41.223774 622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:30:41.233323 627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:41.233336 628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:41.233314 632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:41.233449 622 server_base.cc:1034] running on GCE node
I20260808 22:30:41.234037 622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:30:41.234632 622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:30:41.235833 622 hybrid_clock.cc:648] HybridClock initialized: now 1786228241235801 us; error 40 us; skew 500 ppm
Aug 08 22:30:41 dist-test-slave-7f01 krb5kdc[455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228241, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:30:41.238552 622 init.cc:377] Logged in from keytab as kudu/127.5.11.193@KRBTEST.COM (short username kudu)
I20260808 22:30:41.239423 622 webserver.cc:460] Webserver started at http://127.5.11.193:32933/ using document root <none> and password file <none>
I20260808 22:30:41.240073 622 fs_manager.cc:359] Metadata directory not provided
I20260808 22:30:41.240139 622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:30:41.240381 622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:30:41.242331 622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-0/data/instance:
uuid: "a98441325f604cf39a13d73abbf64af4"
format_stamp: "Formatted at 2026-08-08 22:30:41 on dist-test-slave-7f01"
server_key: "9f5b758bb8e9f0e5d9b0153ed7b26b71"
server_key_iv: "d1c9789d4a5a4cbbe210da787ddcb149"
server_key_version: "encryptionkey@0"
I20260808 22:30:41.242909 622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance:
uuid: "a98441325f604cf39a13d73abbf64af4"
format_stamp: "Formatted at 2026-08-08 22:30:41 on dist-test-slave-7f01"
server_key: "9f5b758bb8e9f0e5d9b0153ed7b26b71"
server_key_iv: "d1c9789d4a5a4cbbe210da787ddcb149"
server_key_version: "encryptionkey@0"
I20260808 22:30:41.246462 622 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260808 22:30:41.248836 640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:41.250059 622 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:30:41.250209 622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-0/wal
uuid: "a98441325f604cf39a13d73abbf64af4"
format_stamp: "Formatted at 2026-08-08 22:30:41 on dist-test-slave-7f01"
server_key: "9f5b758bb8e9f0e5d9b0153ed7b26b71"
server_key_iv: "d1c9789d4a5a4cbbe210da787ddcb149"
server_key_version: "encryptionkey@0"
I20260808 22:30:41.250320 622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:30:41.262528 622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:30:41.265482 622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:30:41.265682 622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:30:41.266260 622 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:30:41.267115 622 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:30:41.267189 622 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:41.267258 622 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:30:41.267300 622 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:41.276528 622 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.193:38253
I20260808 22:30:41.276571 808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.193:38253 every 8 connection(s)
I20260808 22:30:41.277400 622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
I20260808 22:30:41.285271 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 622
I20260808 22:30:41.285403 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance
I20260808 22:30:41.285686 5167 external_mini_cluster.cc:1428] Setting key b5715fa192c3dacff39a3f14fd98415b
Aug 08 22:30:41 dist-test-slave-7f01 krb5kdc[455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228241, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:30:41.291705 587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:41.279080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:48303 (local address 127.5.11.254:34267)
0808 22:30:41.279210 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:41.279218 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:41.280140 (+ 922us) server_negotiation.cc:407] Connection header received
0808 22:30:41.281516 (+ 1376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:41.281519 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:41.281579 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:41.281681 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:41.282867 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:41.283404 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:41.284616 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:41.284848 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:41.284956 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:41.287869 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:41.287872 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:41.287874 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:30:41.287897 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:41.289517 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:41.290031 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:41.290036 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:41.290039 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:41.290139 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:41.290468 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:41.290474 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:41.290478 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:41.290729 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:41.290849 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:41.291478 (+ 629us) server_negotiation.cc:299] Negotiation successful
0808 22:30:41.291584 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 22:30:41.292966 815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:41.279485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:34267 (local address 127.5.11.193:48303)
0808 22:30:41.279995 (+ 510us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:41.280038 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:41.281289 (+ 1251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:41.281872 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:41.281879 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:41.282232 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:30:41.282709 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:41.282721 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:41.283536 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:41.283539 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:41.284481 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:41.284492 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:41.284999 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:41.285003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:41.285197 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:41.285827 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:30:41.285845 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:30:41.287609 (+ 1764us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:30:41.289654 (+ 2045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:30:41.289658 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:30:41.289669 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:30:41.289923 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:30:41.290237 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:30:41.290239 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:30:41.290241 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:30:41.290374 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:30:41.290873 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:30:41.290883 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:30:41.291215 (+ 332us) client_negotiation.cc:769] Sending connection context
0808 22:30:41.291434 (+ 219us) client_negotiation.cc:240] Negotiation successful
0808 22:30:41.291823 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":163,"threads_started":1}
I20260808 22:30:41.294338 809 heartbeater.cc:347] Connected to a master server at 127.5.11.254:34267
I20260808 22:30:41.294585 809 heartbeater.cc:464] Registering TS with master...
I20260808 22:30:41.295092 809 heartbeater.cc:511] Master 127.5.11.254:34267 requested a full tablet report, sending...
I20260808 22:30:41.296617 529 ts_manager.cc:194] Registered new tserver with Master: a98441325f604cf39a13d73abbf64af4 (127.5.11.193:38253)
WARNING: no policy specified for kudu/127.5.11.194@KRBTEST.COM; defaulting to no policy
I20260808 22:30:41.297775 529 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.193@KRBTEST.COM'} at 127.5.11.193:48303
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab.
Entry for principal kudu/127.5.11.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab.
I20260808 22:30:41.315147 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.194:0
--local_ip_for_outbound_sockets=127.5.11.194
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:34267
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:44607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:30:41.427306 818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:30:41.427584 818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:30:41.427659 818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:30:41.431624 818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:30:41.431704 818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:30:41.431828 818 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.194
I20260808 22:30:41.437055 818 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:44607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:34267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.194
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:30:41.438048 818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:30:41.439095 818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:30:41.448601 827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:41.448616 824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:41.448625 825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:41.448942 818 server_base.cc:1034] running on GCE node
I20260808 22:30:41.449419 818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:30:41.450160 818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:30:41.451352 818 hybrid_clock.cc:648] HybridClock initialized: now 1786228241451320 us; error 44 us; skew 500 ppm
Aug 08 22:30:41 dist-test-slave-7f01 krb5kdc[455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228241, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:30:41.454352 818 init.cc:377] Logged in from keytab as kudu/127.5.11.194@KRBTEST.COM (short username kudu)
I20260808 22:30:41.455220 818 webserver.cc:460] Webserver started at http://127.5.11.194:42647/ using document root <none> and password file <none>
I20260808 22:30:41.455839 818 fs_manager.cc:359] Metadata directory not provided
I20260808 22:30:41.455910 818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:30:41.456136 818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:30:41.458268 818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-1/data/instance:
uuid: "c113587ca56f44f089442be95ad96817"
format_stamp: "Formatted at 2026-08-08 22:30:41 on dist-test-slave-7f01"
server_key: "ae0cb0e792e6448684e6c76f3ff75fa2"
server_key_iv: "d18f847a9d746c289e8caa884b1003bc"
server_key_version: "encryptionkey@0"
I20260808 22:30:41.458945 818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance:
uuid: "c113587ca56f44f089442be95ad96817"
format_stamp: "Formatted at 2026-08-08 22:30:41 on dist-test-slave-7f01"
server_key: "ae0cb0e792e6448684e6c76f3ff75fa2"
server_key_iv: "d18f847a9d746c289e8caa884b1003bc"
server_key_version: "encryptionkey@0"
I20260808 22:30:41.462853 818 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 22:30:41.465332 836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:41.466563 818 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:30:41.466692 818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-1/wal
uuid: "c113587ca56f44f089442be95ad96817"
format_stamp: "Formatted at 2026-08-08 22:30:41 on dist-test-slave-7f01"
server_key: "ae0cb0e792e6448684e6c76f3ff75fa2"
server_key_iv: "d18f847a9d746c289e8caa884b1003bc"
server_key_version: "encryptionkey@0"
I20260808 22:30:41.466827 818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:30:41.479194 818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:30:41.482445 818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:30:41.482647 818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:30:41.483230 818 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:30:41.484113 818 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:30:41.484185 818 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:41.484254 818 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:30:41.484295 818 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:41.493320 818 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.194:38391
I20260808 22:30:41.493340 1016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.194:38391 every 8 connection(s)
I20260808 22:30:41.494240 818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
I20260808 22:30:41.501514 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 818
I20260808 22:30:41.501660 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance
I20260808 22:30:41.501986 5167 external_mini_cluster.cc:1428] Setting key 84269acdb8cc6eacaecced4515dd7588
Aug 08 22:30:41 dist-test-slave-7f01 krb5kdc[455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228241, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:30:41.508790 587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:41.496070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:41291 (local address 127.5.11.254:34267)
0808 22:30:41.496202 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:41.496205 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:41.497083 (+ 878us) server_negotiation.cc:407] Connection header received
0808 22:30:41.498474 (+ 1391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:41.498478 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:41.498541 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:41.498637 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:41.499851 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:41.500391 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:41.501555 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:41.501858 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:41.501972 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:41.504922 (+ 2950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:41.504928 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:41.504934 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:30:41.504964 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:41.506609 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:41.507141 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:41.507145 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:41.507147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:41.507237 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:41.507598 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:41.507601 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:41.507603 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:41.507861 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:41.507974 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:41.508491 (+ 517us) server_negotiation.cc:299] Negotiation successful
0808 22:30:41.508621 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 22:30:41.509649 1023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:41.496441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:34267 (local address 127.5.11.194:41291)
0808 22:30:41.496910 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:41.496954 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:41.498242 (+ 1288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:41.498820 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:41.498827 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:41.499190 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:30:41.499672 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:41.499685 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:41.500520 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:41.500524 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:41.501422 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:41.501432 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:41.502016 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:41.502021 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:41.502193 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:41.502830 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:30:41.502851 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:30:41.504746 (+ 1895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:30:41.506730 (+ 1984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:30:41.506736 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:30:41.506750 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:30:41.507032 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:30:41.507340 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:30:41.507343 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:30:41.507345 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:30:41.507495 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:30:41.507972 (+ 477us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:30:41.507979 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:30:41.508231 (+ 252us) client_negotiation.cc:769] Sending connection context
0808 22:30:41.508439 (+ 208us) client_negotiation.cc:240] Negotiation successful
0808 22:30:41.508771 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":122,"threads_started":1}
I20260808 22:30:41.510954 1017 heartbeater.cc:347] Connected to a master server at 127.5.11.254:34267
I20260808 22:30:41.511197 1017 heartbeater.cc:464] Registering TS with master...
I20260808 22:30:41.511699 1017 heartbeater.cc:511] Master 127.5.11.254:34267 requested a full tablet report, sending...
I20260808 22:30:41.512799 529 ts_manager.cc:194] Registered new tserver with Master: c113587ca56f44f089442be95ad96817 (127.5.11.194:38391)
WARNING: no policy specified for kudu/127.5.11.195@KRBTEST.COM; defaulting to no policy
I20260808 22:30:41.513401 529 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.194@KRBTEST.COM'} at 127.5.11.194:41291
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab.
Entry for principal kudu/127.5.11.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab.
I20260808 22:30:41.531211 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.195:0
--local_ip_for_outbound_sockets=127.5.11.195
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:34267
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:44607
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:30:41.642567 1026 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:30:41.642848 1026 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:30:41.642920 1026 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:30:41.646844 1026 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:30:41.646929 1026 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:30:41.647022 1026 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.195
I20260808 22:30:41.652050 1026 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:44607
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:34267
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1026
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.195
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:30:41.653076 1026 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:30:41.654134 1026 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:30:41.663898 1035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:41.663956 1033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:41.663900 1039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:41.664213 1026 server_base.cc:1034] running on GCE node
I20260808 22:30:41.664687 1026 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:30:41.665343 1026 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:30:41.666531 1026 hybrid_clock.cc:648] HybridClock initialized: now 1786228241666514 us; error 27 us; skew 500 ppm
Aug 08 22:30:41 dist-test-slave-7f01 krb5kdc[455](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228241, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:30:41.669358 1026 init.cc:377] Logged in from keytab as kudu/127.5.11.195@KRBTEST.COM (short username kudu)
I20260808 22:30:41.670270 1026 webserver.cc:460] Webserver started at http://127.5.11.195:41279/ using document root <none> and password file <none>
I20260808 22:30:41.670960 1026 fs_manager.cc:359] Metadata directory not provided
I20260808 22:30:41.671049 1026 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:30:41.671262 1026 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:30:41.673460 1026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-2/data/instance:
uuid: "d56825bd28ec4155bd690003ad6c49e8"
format_stamp: "Formatted at 2026-08-08 22:30:41 on dist-test-slave-7f01"
server_key: "8cc05a8cac82284d23f3ee67df9ac7cc"
server_key_iv: "288850f80caf95348c62eabd8b8bf65b"
server_key_version: "encryptionkey@0"
I20260808 22:30:41.674022 1026 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance:
uuid: "d56825bd28ec4155bd690003ad6c49e8"
format_stamp: "Formatted at 2026-08-08 22:30:41 on dist-test-slave-7f01"
server_key: "8cc05a8cac82284d23f3ee67df9ac7cc"
server_key_iv: "288850f80caf95348c62eabd8b8bf65b"
server_key_version: "encryptionkey@0"
I20260808 22:30:41.677728 1026 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 22:30:41.680163 1048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:41.681332 1026 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:30:41.681473 1026 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-2/wal
uuid: "d56825bd28ec4155bd690003ad6c49e8"
format_stamp: "Formatted at 2026-08-08 22:30:41 on dist-test-slave-7f01"
server_key: "8cc05a8cac82284d23f3ee67df9ac7cc"
server_key_iv: "288850f80caf95348c62eabd8b8bf65b"
server_key_version: "encryptionkey@0"
I20260808 22:30:41.681584 1026 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:30:41.693090 1026 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:30:41.696270 1026 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:30:41.696471 1026 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:30:41.697021 1026 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:30:41.697847 1026 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:30:41.697918 1026 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:41.697991 1026 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:30:41.698032 1026 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:41.706969 1026 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.195:46793
I20260808 22:30:41.707001 1218 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.195:46793 every 8 connection(s)
I20260808 22:30:41.707937 1026 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
Aug 08 22:30:41 dist-test-slave-7f01 krb5kdc[455](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228241, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:30:41.717567 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 1026
I20260808 22:30:41.717724 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance
I20260808 22:30:41.718047 5167 external_mini_cluster.cc:1428] Setting key a6ea70a686a8026709d9c44df5b0ede6
I20260808 22:30:41.721513 587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:41.709727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:44611 (local address 127.5.11.254:34267)
0808 22:30:41.709862 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:41.709866 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:41.710599 (+ 733us) server_negotiation.cc:407] Connection header received
0808 22:30:41.711984 (+ 1385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:41.711987 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:41.712052 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:41.712148 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:41.713278 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:41.713810 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:41.714940 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:41.715166 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:41.715273 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:41.717654 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:41.717658 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:41.717661 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:30:41.717705 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:41.719469 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:41.720023 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:41.720027 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:41.720028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:41.720126 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:41.720517 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:41.720519 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:41.720521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:41.720694 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:41.720770 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:41.721227 (+ 457us) server_negotiation.cc:299] Negotiation successful
0808 22:30:41.721325 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 22:30:41.722225 1221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:41.710024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:34267 (local address 127.5.11.195:44611)
0808 22:30:41.710449 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:41.710490 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:41.711718 (+ 1228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:41.712322 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:41.712329 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:41.712664 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:30:41.713131 (+ 467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:41.713144 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:41.713940 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:41.713944 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:41.714807 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:41.714817 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:41.715312 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:41.715316 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:41.715486 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:41.716077 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:30:41.716093 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:30:41.717496 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:30:41.719586 (+ 2090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:30:41.719591 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:30:41.719603 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:30:41.719929 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:30:41.720225 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:30:41.720228 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:30:41.720229 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:30:41.720420 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:30:41.720777 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:30:41.720782 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:30:41.721007 (+ 225us) client_negotiation.cc:769] Sending connection context
0808 22:30:41.721155 (+ 148us) client_negotiation.cc:240] Negotiation successful
0808 22:30:41.721404 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":117,"threads_started":1}
I20260808 22:30:41.723381 1219 heartbeater.cc:347] Connected to a master server at 127.5.11.254:34267
I20260808 22:30:41.723767 1219 heartbeater.cc:464] Registering TS with master...
I20260808 22:30:41.724299 1219 heartbeater.cc:511] Master 127.5.11.254:34267 requested a full tablet report, sending...
I20260808 22:30:41.725349 529 ts_manager.cc:194] Registered new tserver with Master: d56825bd28ec4155bd690003ad6c49e8 (127.5.11.195:46793)
I20260808 22:30:41.725950 529 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.195@KRBTEST.COM'} at 127.5.11.195:44611
I20260808 22:30:41.731949 5167 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:30:41.741250 587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:41.734059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36708 (local address 127.5.11.254:34267)
0808 22:30:41.734198 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:41.734204 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:41.734424 (+ 220us) server_negotiation.cc:407] Connection header received
0808 22:30:41.734507 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:41.734509 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:41.734546 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:41.734610 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:41.735354 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:41.735958 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:41.736789 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:41.737011 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:41.737093 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:41.738342 (+ 1249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:41.738345 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:41.738347 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:30:41.738368 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:41.739803 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:41.740174 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:41.740177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:41.740178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:41.740250 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:41.740474 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:41.740476 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:41.740477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:41.740670 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:41.740770 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:41.740975 (+ 205us) server_negotiation.cc:299] Negotiation successful
0808 22:30:41.741064 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 22:30:41.744055 529 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36708:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:30:41.747159 529 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:30:41.761492 1237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:41.755366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:46793 (local address 127.0.0.1:40974)
0808 22:30:41.756142 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:41.756157 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:41.756244 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:41.757109 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:41.757118 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:41.757144 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:30:41.757275 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:41.757282 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:41.758736 (+ 1454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:41.758739 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:41.759900 (+ 1161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:41.759917 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:41.760914 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:41.760917 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:41.761027 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:41.761066 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 22:30:41.761174 (+ 108us) client_negotiation.cc:240] Negotiation successful
0808 22:30:41.761318 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"mutex_wait_us":2,"thread_start_us":81,"threads_started":1}
I20260808 22:30:41.761679 1235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:41.755273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:38391 (local address 127.0.0.1:59592)
0808 22:30:41.755946 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:41.755995 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:41.756129 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:41.757492 (+ 1363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:41.757497 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:41.757517 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:30:41.757602 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:41.757609 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:41.758907 (+ 1298us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:41.758913 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:41.760184 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:41.760196 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:41.761237 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:41.761241 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:41.761378 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:41.761401 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 22:30:41.761475 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 22:30:41.761555 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"mutex_wait_us":52,"thread_start_us":82,"threads_started":1}
I20260808 22:30:41.761792 1239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:41.755406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59592 (local address 127.5.11.194:38391)
0808 22:30:41.757016 (+ 1610us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:41.757029 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:41.757051 (+ 22us) server_negotiation.cc:407] Connection header received
0808 22:30:41.757121 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:41.757125 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:41.757298 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:41.757510 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:30:41.757703 (+ 193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:41.758771 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:41.760317 (+ 1546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:41.761133 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:41.761236 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:41.761373 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:41.761529 (+ 156us) server_negotiation.cc:299] Negotiation successful
0808 22:30:41.761647 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1503,"thread_start_us":119,"threads_started":1}
I20260808 22:30:41.762194 1240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:41.755640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:38253 (local address 127.0.0.1:37642)
0808 22:30:41.756666 (+ 1026us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:41.756679 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:41.756761 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:41.757343 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:41.757349 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:41.757370 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:30:41.757471 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:41.757479 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:41.758527 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:41.758533 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:41.760184 (+ 1651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:41.760196 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:41.761122 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:41.761125 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:41.761228 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:41.761257 (+ 29us) client_negotiation.cc:769] Sending connection context
0808 22:30:41.761332 (+ 75us) client_negotiation.cc:240] Negotiation successful
0808 22:30:41.761418 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":965,"mutex_wait_us":124,"thread_start_us":823,"threads_started":1}
I20260808 22:30:41.762328 1241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:41.755960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37642 (local address 127.5.11.193:38253)
0808 22:30:41.756755 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:41.756770 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:41.756790 (+ 20us) server_negotiation.cc:407] Connection header received
0808 22:30:41.756873 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:41.756877 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:41.757015 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:41.757184 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:30:41.757600 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:41.758401 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:41.760342 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:41.760901 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:41.761005 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:41.761148 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:41.762053 (+ 905us) server_negotiation.cc:299] Negotiation successful
0808 22:30:41.762164 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":708,"thread_start_us":80,"threads_started":1}
I20260808 22:30:41.763597 1242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:41.756288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40974 (local address 127.5.11.195:46793)
0808 22:30:41.756579 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:41.756591 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:41.756616 (+ 25us) server_negotiation.cc:407] Connection header received
0808 22:30:41.756699 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:41.756707 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:41.756936 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:41.757102 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:30:41.757402 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:41.758589 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:41.760025 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:41.760773 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:41.760885 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:41.761034 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:41.763340 (+ 2306us) server_negotiation.cc:299] Negotiation successful
0808 22:30:41.763456 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":114,"threads_started":1}
I20260808 22:30:41.764883 913 tablet_service.cc:1467] Processing CreateTablet for tablet a042eb2e3ff540fcae67f04d3e30d9c3 (DEFAULT_TABLE table=test-table [id=4989acc486cf4d13ba5f5f6d3658392f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:30:41.765285 709 tablet_service.cc:1467] Processing CreateTablet for tablet a042eb2e3ff540fcae67f04d3e30d9c3 (DEFAULT_TABLE table=test-table [id=4989acc486cf4d13ba5f5f6d3658392f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:30:41.765930 913 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a042eb2e3ff540fcae67f04d3e30d9c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:30:41.766321 709 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a042eb2e3ff540fcae67f04d3e30d9c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:30:41.766203 1121 tablet_service.cc:1467] Processing CreateTablet for tablet a042eb2e3ff540fcae67f04d3e30d9c3 (DEFAULT_TABLE table=test-table [id=4989acc486cf4d13ba5f5f6d3658392f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:30:41.767167 1121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a042eb2e3ff540fcae67f04d3e30d9c3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:30:41.772387 1243 tablet_bootstrap.cc:492] T a042eb2e3ff540fcae67f04d3e30d9c3 P c113587ca56f44f089442be95ad96817: Bootstrap starting.
I20260808 22:30:41.774019 1244 tablet_bootstrap.cc:492] T a042eb2e3ff540fcae67f04d3e30d9c3 P a98441325f604cf39a13d73abbf64af4: Bootstrap starting.
I20260808 22:30:41.774776 1243 tablet_bootstrap.cc:654] T a042eb2e3ff540fcae67f04d3e30d9c3 P c113587ca56f44f089442be95ad96817: Neither blocks nor log segments found. Creating new log.
I20260808 22:30:41.775978 1244 tablet_bootstrap.cc:654] T a042eb2e3ff540fcae67f04d3e30d9c3 P a98441325f604cf39a13d73abbf64af4: Neither blocks nor log segments found. Creating new log.
I20260808 22:30:41.776003 1243 log.cc:824] T a042eb2e3ff540fcae67f04d3e30d9c3 P c113587ca56f44f089442be95ad96817: Log is configured to *not* fsync() on all Append() calls
I20260808 22:30:41.776115 1246 tablet_bootstrap.cc:492] T a042eb2e3ff540fcae67f04d3e30d9c3 P d56825bd28ec4155bd690003ad6c49e8: Bootstrap starting.
I20260808 22:30:41.777266 1244 log.cc:824] T a042eb2e3ff540fcae67f04d3e30d9c3 P a98441325f604cf39a13d73abbf64af4: Log is configured to *not* fsync() on all Append() calls
I20260808 22:30:41.778175 1243 tablet_bootstrap.cc:492] T a042eb2e3ff540fcae67f04d3e30d9c3 P c113587ca56f44f089442be95ad96817: No bootstrap required, opened a new log
I20260808 22:30:41.778203 1246 tablet_bootstrap.cc:654] T a042eb2e3ff540fcae67f04d3e30d9c3 P d56825bd28ec4155bd690003ad6c49e8: Neither blocks nor log segments found. Creating new log.
I20260808 22:30:41.778388 1243 ts_tablet_manager.cc:1397] T a042eb2e3ff540fcae67f04d3e30d9c3 P c113587ca56f44f089442be95ad96817: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260808 22:30:41.779349 1244 tablet_bootstrap.cc:492] T a042eb2e3ff540fcae67f04d3e30d9c3 P a98441325f604cf39a13d73abbf64af4: No bootstrap required, opened a new log
I20260808 22:30:41.779420 1246 log.cc:824] T a042eb2e3ff540fcae67f04d3e30d9c3 P d56825bd28ec4155bd690003ad6c49e8: Log is configured to *not* fsync() on all Append() calls
I20260808 22:30:41.779546 1244 ts_tablet_manager.cc:1397] T a042eb2e3ff540fcae67f04d3e30d9c3 P a98441325f604cf39a13d73abbf64af4: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260808 22:30:41.781347 1246 tablet_bootstrap.cc:492] T a042eb2e3ff540fcae67f04d3e30d9c3 P d56825bd28ec4155bd690003ad6c49e8: No bootstrap required, opened a new log
I20260808 22:30:41.781535 1246 ts_tablet_manager.cc:1397] T a042eb2e3ff540fcae67f04d3e30d9c3 P d56825bd28ec4155bd690003ad6c49e8: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260808 22:30:41.781628 1243 raft_consensus.cc:348] T a042eb2e3ff540fcae67f04d3e30d9c3 P c113587ca56f44f089442be95ad96817 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c113587ca56f44f089442be95ad96817" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 38391 } } peers { permanent_uuid: "d56825bd28ec4155bd690003ad6c49e8" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 46793 } } peers { permanent_uuid: "a98441325f604cf39a13d73abbf64af4" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 38253 } }
I20260808 22:30:41.781888 1243 raft_consensus.cc:374] T a042eb2e3ff540fcae67f04d3e30d9c3 P c113587ca56f44f089442be95ad96817 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:30:41.781955 1243 raft_consensus.cc:729] T a042eb2e3ff540fcae67f04d3e30d9c3 P c113587ca56f44f089442be95ad96817 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c113587ca56f44f089442be95ad96817, State: Initialized, Role: FOLLOWER
I20260808 22:30:41.782395 1243 consensus_queue.cc:260] T a042eb2e3ff540fcae67f04d3e30d9c3 P c113587ca56f44f089442be95ad96817 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c113587ca56f44f089442be95ad96817" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 38391 } } peers { permanent_uuid: "d56825bd28ec4155bd690003ad6c49e8" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 46793 } } peers { permanent_uuid: "a98441325f604cf39a13d73abbf64af4" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 38253 } }
I20260808 22:30:41.783010 1017 heartbeater.cc:503] Master 127.5.11.254:34267 was elected leader, sending a full tablet report...
I20260808 22:30:41.783361 1243 ts_tablet_manager.cc:1428] T a042eb2e3ff540fcae67f04d3e30d9c3 P c113587ca56f44f089442be95ad96817: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260808 22:30:41.783267 1244 raft_consensus.cc:348] T a042eb2e3ff540fcae67f04d3e30d9c3 P a98441325f604cf39a13d73abbf64af4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c113587ca56f44f089442be95ad96817" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 38391 } } peers { permanent_uuid: "d56825bd28ec4155bd690003ad6c49e8" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 46793 } } peers { permanent_uuid: "a98441325f604cf39a13d73abbf64af4" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 38253 } }
I20260808 22:30:41.783519 1244 raft_consensus.cc:374] T a042eb2e3ff540fcae67f04d3e30d9c3 P a98441325f604cf39a13d73abbf64af4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:30:41.783568 1244 raft_consensus.cc:729] T a042eb2e3ff540fcae67f04d3e30d9c3 P a98441325f604cf39a13d73abbf64af4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a98441325f604cf39a13d73abbf64af4, State: Initialized, Role: FOLLOWER
I20260808 22:30:41.784029 1244 consensus_queue.cc:260] T a042eb2e3ff540fcae67f04d3e30d9c3 P a98441325f604cf39a13d73abbf64af4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c113587ca56f44f089442be95ad96817" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 38391 } } peers { permanent_uuid: "d56825bd28ec4155bd690003ad6c49e8" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 46793 } } peers { permanent_uuid: "a98441325f604cf39a13d73abbf64af4" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 38253 } }
I20260808 22:30:41.785323 1246 raft_consensus.cc:348] T a042eb2e3ff540fcae67f04d3e30d9c3 P d56825bd28ec4155bd690003ad6c49e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c113587ca56f44f089442be95ad96817" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 38391 } } peers { permanent_uuid: "d56825bd28ec4155bd690003ad6c49e8" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 46793 } } peers { permanent_uuid: "a98441325f604cf39a13d73abbf64af4" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 38253 } }
I20260808 22:30:41.785602 1246 raft_consensus.cc:374] T a042eb2e3ff540fcae67f04d3e30d9c3 P d56825bd28ec4155bd690003ad6c49e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:30:41.785663 1246 raft_consensus.cc:729] T a042eb2e3ff540fcae67f04d3e30d9c3 P d56825bd28ec4155bd690003ad6c49e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d56825bd28ec4155bd690003ad6c49e8, State: Initialized, Role: FOLLOWER
I20260808 22:30:41.786078 1246 consensus_queue.cc:260] T a042eb2e3ff540fcae67f04d3e30d9c3 P d56825bd28ec4155bd690003ad6c49e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c113587ca56f44f089442be95ad96817" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 38391 } } peers { permanent_uuid: "d56825bd28ec4155bd690003ad6c49e8" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 46793 } } peers { permanent_uuid: "a98441325f604cf39a13d73abbf64af4" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 38253 } }
I20260808 22:30:41.786540 809 heartbeater.cc:503] Master 127.5.11.254:34267 was elected leader, sending a full tablet report...
I20260808 22:30:41.786857 1244 ts_tablet_manager.cc:1428] T a042eb2e3ff540fcae67f04d3e30d9c3 P a98441325f604cf39a13d73abbf64af4: Time spent starting tablet: real 0.007s user 0.003s sys 0.003s
I20260808 22:30:41.787030 1246 ts_tablet_manager.cc:1428] T a042eb2e3ff540fcae67f04d3e30d9c3 P d56825bd28ec4155bd690003ad6c49e8: Time spent starting tablet: real 0.005s user 0.005s sys 0.002s
I20260808 22:30:41.787788 1219 heartbeater.cc:503] Master 127.5.11.254:34267 was elected leader, sending a full tablet report...
I20260808 22:30:41.845472 1254 raft_consensus.cc:482] T a042eb2e3ff540fcae67f04d3e30d9c3 P d56825bd28ec4155bd690003ad6c49e8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:30:41.845676 1254 raft_consensus.cc:504] T a042eb2e3ff540fcae67f04d3e30d9c3 P d56825bd28ec4155bd690003ad6c49e8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c113587ca56f44f089442be95ad96817" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 38391 } } peers { permanent_uuid: "d56825bd28ec4155bd690003ad6c49e8" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 46793 } } peers { permanent_uuid: "a98441325f604cf39a13d73abbf64af4" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 38253 } }
I20260808 22:30:41.846881 1254 leader_election.cc:288] T a042eb2e3ff540fcae67f04d3e30d9c3 P d56825bd28ec4155bd690003ad6c49e8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c113587ca56f44f089442be95ad96817 (127.5.11.194:38391), a98441325f604cf39a13d73abbf64af4 (127.5.11.193:38253)
I20260808 22:30:41.850775 1221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:41.847138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:38253 (local address 127.5.11.195:47639)
0808 22:30:41.847349 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:41.847364 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:41.847457 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:41.847794 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:41.847797 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:41.847815 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:30:41.847935 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:41.847940 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:41.848677 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:41.848683 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:41.849530 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:41.849540 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:41.850352 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:41.850356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:41.850463 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:41.850481 (+ 18us) client_negotiation.cc:769] Sending connection context
0808 22:30:41.850572 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 22:30:41.850641 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":42}
I20260808 22:30:41.850775 1241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:41.847292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:47639 (local address 127.5.11.193:38253)
0808 22:30:41.847544 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:41.847554 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:41.847567 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:30:41.847618 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:41.847620 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:41.847657 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:41.847741 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:30:41.848060 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:41.848564 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:41.849660 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:41.850169 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:41.850295 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:41.850360 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:41.850592 (+ 232us) server_negotiation.cc:299] Negotiation successful
0808 22:30:41.850662 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149}
I20260808 22:30:41.851024 1239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:41.847251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:41881 (local address 127.5.11.194:38391)
0808 22:30:41.847415 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:41.847420 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:41.847624 (+ 204us) server_negotiation.cc:407] Connection header received
0808 22:30:41.847801 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:41.847804 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:41.847852 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:41.847934 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:30:41.848202 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:41.848679 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:41.849859 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:41.850436 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:41.850568 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:41.850625 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:41.850802 (+ 177us) server_negotiation.cc:299] Negotiation successful
0808 22:30:41.850892 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260808 22:30:41.851332 1258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:41.847136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:38391 (local address 127.5.11.195:41881)
0808 22:30:41.847526 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:41.847538 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:41.847666 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:41.847944 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:41.847947 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:41.847964 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:30:41.848093 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:41.848098 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:41.848788 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:41.848794 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:41.849682 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:41.849694 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:41.850563 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:41.850567 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:41.850675 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:41.850702 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 22:30:41.850787 (+ 85us) client_negotiation.cc:240] Negotiation successful
0808 22:30:41.851228 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":22,"thread_start_us":141,"threads_started":1}
I20260808 22:30:41.851454 741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a042eb2e3ff540fcae67f04d3e30d9c3" candidate_uuid: "d56825bd28ec4155bd690003ad6c49e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a98441325f604cf39a13d73abbf64af4" is_pre_election: true
I20260808 22:30:41.851810 741 raft_consensus.cc:2454] T a042eb2e3ff540fcae67f04d3e30d9c3 P a98441325f604cf39a13d73abbf64af4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d56825bd28ec4155bd690003ad6c49e8 in term 0.
I20260808 22:30:41.852043 941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a042eb2e3ff540fcae67f04d3e30d9c3" candidate_uuid: "d56825bd28ec4155bd690003ad6c49e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c113587ca56f44f089442be95ad96817" is_pre_election: true
I20260808 22:30:41.852314 941 raft_consensus.cc:2454] T a042eb2e3ff540fcae67f04d3e30d9c3 P c113587ca56f44f089442be95ad96817 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d56825bd28ec4155bd690003ad6c49e8 in term 0.
I20260808 22:30:41.852322 1049 leader_election.cc:302] T a042eb2e3ff540fcae67f04d3e30d9c3 P d56825bd28ec4155bd690003ad6c49e8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a98441325f604cf39a13d73abbf64af4, d56825bd28ec4155bd690003ad6c49e8; no voters:
I20260808 22:30:41.852581 1254 raft_consensus.cc:2781] T a042eb2e3ff540fcae67f04d3e30d9c3 P d56825bd28ec4155bd690003ad6c49e8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:30:41.852676 1254 raft_consensus.cc:482] T a042eb2e3ff540fcae67f04d3e30d9c3 P d56825bd28ec4155bd690003ad6c49e8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:30:41.852733 1254 raft_consensus.cc:3037] T a042eb2e3ff540fcae67f04d3e30d9c3 P d56825bd28ec4155bd690003ad6c49e8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:30:41.853844 1254 raft_consensus.cc:504] T a042eb2e3ff540fcae67f04d3e30d9c3 P d56825bd28ec4155bd690003ad6c49e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c113587ca56f44f089442be95ad96817" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 38391 } } peers { permanent_uuid: "d56825bd28ec4155bd690003ad6c49e8" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 46793 } } peers { permanent_uuid: "a98441325f604cf39a13d73abbf64af4" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 38253 } }
I20260808 22:30:41.854234 1254 leader_election.cc:288] T a042eb2e3ff540fcae67f04d3e30d9c3 P d56825bd28ec4155bd690003ad6c49e8 [CANDIDATE]: Term 1 election: Requested vote from peers c113587ca56f44f089442be95ad96817 (127.5.11.194:38391), a98441325f604cf39a13d73abbf64af4 (127.5.11.193:38253)
I20260808 22:30:41.854633 741 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a042eb2e3ff540fcae67f04d3e30d9c3" candidate_uuid: "d56825bd28ec4155bd690003ad6c49e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a98441325f604cf39a13d73abbf64af4"
I20260808 22:30:41.854759 741 raft_consensus.cc:3037] T a042eb2e3ff540fcae67f04d3e30d9c3 P a98441325f604cf39a13d73abbf64af4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:30:41.854758 941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a042eb2e3ff540fcae67f04d3e30d9c3" candidate_uuid: "d56825bd28ec4155bd690003ad6c49e8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c113587ca56f44f089442be95ad96817"
I20260808 22:30:41.854923 941 raft_consensus.cc:3037] T a042eb2e3ff540fcae67f04d3e30d9c3 P c113587ca56f44f089442be95ad96817 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:30:41.855909 741 raft_consensus.cc:2454] T a042eb2e3ff540fcae67f04d3e30d9c3 P a98441325f604cf39a13d73abbf64af4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d56825bd28ec4155bd690003ad6c49e8 in term 1.
I20260808 22:30:41.856035 941 raft_consensus.cc:2454] T a042eb2e3ff540fcae67f04d3e30d9c3 P c113587ca56f44f089442be95ad96817 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d56825bd28ec4155bd690003ad6c49e8 in term 1.
I20260808 22:30:41.856244 1049 leader_election.cc:302] T a042eb2e3ff540fcae67f04d3e30d9c3 P d56825bd28ec4155bd690003ad6c49e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a98441325f604cf39a13d73abbf64af4, d56825bd28ec4155bd690003ad6c49e8; no voters:
I20260808 22:30:41.856462 1254 raft_consensus.cc:2781] T a042eb2e3ff540fcae67f04d3e30d9c3 P d56825bd28ec4155bd690003ad6c49e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:30:41.856691 1254 raft_consensus.cc:686] T a042eb2e3ff540fcae67f04d3e30d9c3 P d56825bd28ec4155bd690003ad6c49e8 [term 1 LEADER]: Becoming Leader. State: Replica: d56825bd28ec4155bd690003ad6c49e8, State: Running, Role: LEADER
I20260808 22:30:41.857039 1254 consensus_queue.cc:237] T a042eb2e3ff540fcae67f04d3e30d9c3 P d56825bd28ec4155bd690003ad6c49e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c113587ca56f44f089442be95ad96817" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 38391 } } peers { permanent_uuid: "d56825bd28ec4155bd690003ad6c49e8" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 46793 } } peers { permanent_uuid: "a98441325f604cf39a13d73abbf64af4" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 38253 } }
I20260808 22:30:41.860153 526 catalog_manager.cc:5495] T a042eb2e3ff540fcae67f04d3e30d9c3 P d56825bd28ec4155bd690003ad6c49e8 reported cstate change: term changed from 0 to 1, leader changed from <none> to d56825bd28ec4155bd690003ad6c49e8 (127.5.11.195). New cstate: current_term: 1 leader_uuid: "d56825bd28ec4155bd690003ad6c49e8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c113587ca56f44f089442be95ad96817" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 38391 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d56825bd28ec4155bd690003ad6c49e8" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 46793 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a98441325f604cf39a13d73abbf64af4" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 38253 } health_report { overall_health: UNKNOWN } } }
I20260808 22:30:41.884986 1242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:41.880098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40978 (local address 127.5.11.195:46793)
0808 22:30:41.880297 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:41.880301 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:41.880314 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:30:41.880409 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:41.880415 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:41.880484 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:41.880601 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:30:41.880895 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:41.881557 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:41.882867 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:41.883279 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:41.883397 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:41.883922 (+ 525us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:30:41.884515 (+ 593us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:30:41.884648 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:41.884748 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 22:30:41.884823 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260808 22:30:41.890839 741 raft_consensus.cc:1261] T a042eb2e3ff540fcae67f04d3e30d9c3 P a98441325f604cf39a13d73abbf64af4 [term 1 FOLLOWER]: Refusing update from remote peer d56825bd28ec4155bd690003ad6c49e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:30:41.890841 941 raft_consensus.cc:1261] T a042eb2e3ff540fcae67f04d3e30d9c3 P c113587ca56f44f089442be95ad96817 [term 1 FOLLOWER]: Refusing update from remote peer d56825bd28ec4155bd690003ad6c49e8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:30:41.891566 1254 consensus_queue.cc:1035] T a042eb2e3ff540fcae67f04d3e30d9c3 P d56825bd28ec4155bd690003ad6c49e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a98441325f604cf39a13d73abbf64af4" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 38253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:30:41.891785 1260 consensus_queue.cc:1035] T a042eb2e3ff540fcae67f04d3e30d9c3 P d56825bd28ec4155bd690003ad6c49e8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c113587ca56f44f089442be95ad96817" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 38391 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:30:41.899242 1266 mvcc.cc:205] Tried to move back new op lower bound from 7316390878777819136 to 7316390878648995840. Current Snapshot: MvccSnapshot[applied={T|T < 7316390878777819136}]
I20260808 22:30:41.900568 1267 mvcc.cc:205] Tried to move back new op lower bound from 7316390878777819136 to 7316390878648995840. Current Snapshot: MvccSnapshot[applied={T|T < 7316390878777819136}]
I20260808 22:30:41.907441 526 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36708:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:30:41.907578 526 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36708:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:30:41.909849 526 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c8d903cbe00f4b35a288bb5a673ff548: Sending DeleteTablet for 3 replicas of tablet a042eb2e3ff540fcae67f04d3e30d9c3
I20260808 22:30:41.910586 913 tablet_service.cc:1514] Processing DeleteTablet for tablet a042eb2e3ff540fcae67f04d3e30d9c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:30:41 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:59592
I20260808 22:30:41.910658 1121 tablet_service.cc:1514] Processing DeleteTablet for tablet a042eb2e3ff540fcae67f04d3e30d9c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:30:41 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:40974
I20260808 22:30:41.910684 709 tablet_service.cc:1514] Processing DeleteTablet for tablet a042eb2e3ff540fcae67f04d3e30d9c3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:30:41 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:37642
I20260808 22:30:41.911089 1279 tablet_replica.cc:330] T a042eb2e3ff540fcae67f04d3e30d9c3 P d56825bd28ec4155bd690003ad6c49e8: stopping tablet replica
I20260808 22:30:41.911296 1281 tablet_replica.cc:330] T a042eb2e3ff540fcae67f04d3e30d9c3 P a98441325f604cf39a13d73abbf64af4: stopping tablet replica
I20260808 22:30:41.911506 1279 raft_consensus.cc:2229] T a042eb2e3ff540fcae67f04d3e30d9c3 P d56825bd28ec4155bd690003ad6c49e8 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:30:41.911577 1281 raft_consensus.cc:2229] T a042eb2e3ff540fcae67f04d3e30d9c3 P a98441325f604cf39a13d73abbf64af4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:30:41.911839 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 622
I20260808 22:30:41.912053 1279 raft_consensus.cc:2258] T a042eb2e3ff540fcae67f04d3e30d9c3 P d56825bd28ec4155bd690003ad6c49e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:30:41.912429 1280 tablet_replica.cc:330] T a042eb2e3ff540fcae67f04d3e30d9c3 P c113587ca56f44f089442be95ad96817: stopping tablet replica
I20260808 22:30:41.912756 1280 raft_consensus.cc:2229] T a042eb2e3ff540fcae67f04d3e30d9c3 P c113587ca56f44f089442be95ad96817 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:30:41.913094 1280 raft_consensus.cc:2258] T a042eb2e3ff540fcae67f04d3e30d9c3 P c113587ca56f44f089442be95ad96817 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:30:41.916208 1279 ts_tablet_manager.cc:1905] T a042eb2e3ff540fcae67f04d3e30d9c3 P d56825bd28ec4155bd690003ad6c49e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:30:41.916486 1280 ts_tablet_manager.cc:1905] T a042eb2e3ff540fcae67f04d3e30d9c3 P c113587ca56f44f089442be95ad96817: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:30:41.919158 1280 ts_tablet_manager.cc:1918] T a042eb2e3ff540fcae67f04d3e30d9c3 P c113587ca56f44f089442be95ad96817: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:30:41.919320 1280 log.cc:1196] T a042eb2e3ff540fcae67f04d3e30d9c3 P c113587ca56f44f089442be95ad96817: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-1/wal/wals/a042eb2e3ff540fcae67f04d3e30d9c3
I20260808 22:30:41.919858 1280 ts_tablet_manager.cc:1939] T a042eb2e3ff540fcae67f04d3e30d9c3 P c113587ca56f44f089442be95ad96817: Deleting consensus metadata
I20260808 22:30:41.919879 1279 ts_tablet_manager.cc:1918] T a042eb2e3ff540fcae67f04d3e30d9c3 P d56825bd28ec4155bd690003ad6c49e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:30:41.919982 1279 log.cc:1196] T a042eb2e3ff540fcae67f04d3e30d9c3 P d56825bd28ec4155bd690003ad6c49e8: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786227991988852-5167-0/minicluster-data/ts-2/wal/wals/a042eb2e3ff540fcae67f04d3e30d9c3
I20260808 22:30:41.920337 1279 ts_tablet_manager.cc:1939] T a042eb2e3ff540fcae67f04d3e30d9c3 P d56825bd28ec4155bd690003ad6c49e8: Deleting consensus metadata
I20260808 22:30:41.921339 505 catalog_manager.cc:4841] TS c113587ca56f44f089442be95ad96817 (127.5.11.194:38391): tablet a042eb2e3ff540fcae67f04d3e30d9c3 (table test-table [id=4989acc486cf4d13ba5f5f6d3658392f]) successfully deleted
I20260808 22:30:41.921778 506 catalog_manager.cc:4841] TS d56825bd28ec4155bd690003ad6c49e8 (127.5.11.195:46793): tablet a042eb2e3ff540fcae67f04d3e30d9c3 (table test-table [id=4989acc486cf4d13ba5f5f6d3658392f]) successfully deleted
W20260808 22:30:41.928148 504 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.11.193:38253 (error 108)
I20260808 22:30:41.928299 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 818
I20260808 22:30:41.931792 1240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:41.931404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:38253 (local address 127.0.0.1:37658)
0808 22:30:41.931586 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:41.931673 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.193:38253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
W20260808 22:30:41.932085 504 catalog_manager.cc:4584] TS a98441325f604cf39a13d73abbf64af4 (127.5.11.193:38253): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a042eb2e3ff540fcae67f04d3e30d9c3: Network error: Client connection negotiation failed: client connection to 127.5.11.193:38253: connect: Connection refused (error 111)
I20260808 22:30:41.938125 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 1026
I20260808 22:30:41.947863 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 474
2026-08-08T22:30:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2554 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-08T22:30:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:30:41Z Disabled control of system clock
I20260808 22:30:41.985644 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:35885
--webserver_interface=127.5.11.254
--webserver_port=0
--builtin_ntp_servers=127.5.11.212:39859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:35885
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 22:30:42.097497 1295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:30:42.097743 1295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:30:42.097777 1295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:30:42.101750 1295 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:30:42.101801 1295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:30:42.101816 1295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:30:42.101833 1295 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:30:42.101848 1295 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:30:42.106366 1295 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:39859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:35885
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:35885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:30:42.107458 1295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:30:42.108785 1295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:30:42.117092 1304 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:42.117146 1307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:42.117121 1295 server_base.cc:1034] running on GCE node
W20260808 22:30:42.117278 1305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:42.117878 1295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:30:42.118845 1295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:30:42.120031 1295 hybrid_clock.cc:648] HybridClock initialized: now 1786228242119999 us; error 44 us; skew 500 ppm
I20260808 22:30:42.121737 1295 webserver.cc:460] Webserver started at http://127.5.11.254:41689/ using document root <none> and password file <none>
I20260808 22:30:42.122298 1295 fs_manager.cc:359] Metadata directory not provided
I20260808 22:30:42.122386 1295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:30:42.122613 1295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:30:42.124637 1295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "67efcefbc91e45dfae35a4db326f5ff8"
format_stamp: "Formatted at 2026-08-08 22:30:42 on dist-test-slave-7f01"
server_key: "895b89120941f43535129e3b9f21491c"
server_key_iv: "299a4db580d1c91fbbfe4c54df0da402"
server_key_version: "encryptionkey@0"
I20260808 22:30:42.125197 1295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "67efcefbc91e45dfae35a4db326f5ff8"
format_stamp: "Formatted at 2026-08-08 22:30:42 on dist-test-slave-7f01"
server_key: "895b89120941f43535129e3b9f21491c"
server_key_iv: "299a4db580d1c91fbbfe4c54df0da402"
server_key_version: "encryptionkey@0"
I20260808 22:30:42.129158 1295 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260808 22:30:42.131428 1315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:42.132582 1295 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 22:30:42.132719 1295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "67efcefbc91e45dfae35a4db326f5ff8"
format_stamp: "Formatted at 2026-08-08 22:30:42 on dist-test-slave-7f01"
server_key: "895b89120941f43535129e3b9f21491c"
server_key_iv: "299a4db580d1c91fbbfe4c54df0da402"
server_key_version: "encryptionkey@0"
I20260808 22:30:42.132824 1295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:30:42.150043 1295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:30:42.150614 1295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:30:42.150796 1295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:30:42.157600 1295 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:35885
I20260808 22:30:42.157624 1401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:35885 every 8 connection(s)
I20260808 22:30:42.158460 1295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:30:42.161306 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 1295
I20260808 22:30:42.161453 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:30:42.161701 1403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:30:42.161787 5167 external_mini_cluster.cc:1428] Setting key a371a338236bde1f1f38b411b50b6336
I20260808 22:30:42.168064 1403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67efcefbc91e45dfae35a4db326f5ff8: Bootstrap starting.
I20260808 22:30:42.170327 1403 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 67efcefbc91e45dfae35a4db326f5ff8: Neither blocks nor log segments found. Creating new log.
I20260808 22:30:42.170396 1412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:42.163009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37984 (local address 127.5.11.254:35885)
0808 22:30:42.163572 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:42.163587 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:42.163620 (+ 33us) server_negotiation.cc:407] Connection header received
0808 22:30:42.164323 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:42.164339 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:42.164701 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:42.165023 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:42.165444 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:42.166374 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:42.167265 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:42.167720 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:42.167947 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:42.168789 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:42.168794 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:42.168805 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:30:42.169166 (+ 361us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:42.169219 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:30:42.169238 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:42.169344 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:42.169496 (+ 152us) server_negotiation.cc:299] Negotiation successful
0808 22:30:42.169744 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":120,"threads_started":1}
I20260808 22:30:42.171137 1403 log.cc:824] T 00000000000000000000000000000000 P 67efcefbc91e45dfae35a4db326f5ff8: Log is configured to *not* fsync() on all Append() calls
I20260808 22:30:42.173074 1403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 67efcefbc91e45dfae35a4db326f5ff8: No bootstrap required, opened a new log
I20260808 22:30:42.176683 1403 raft_consensus.cc:348] T 00000000000000000000000000000000 P 67efcefbc91e45dfae35a4db326f5ff8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67efcefbc91e45dfae35a4db326f5ff8" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 35885 } }
I20260808 22:30:42.176923 1403 raft_consensus.cc:374] T 00000000000000000000000000000000 P 67efcefbc91e45dfae35a4db326f5ff8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:30:42.176997 1403 raft_consensus.cc:729] T 00000000000000000000000000000000 P 67efcefbc91e45dfae35a4db326f5ff8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67efcefbc91e45dfae35a4db326f5ff8, State: Initialized, Role: FOLLOWER
I20260808 22:30:42.177394 1403 consensus_queue.cc:260] T 00000000000000000000000000000000 P 67efcefbc91e45dfae35a4db326f5ff8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67efcefbc91e45dfae35a4db326f5ff8" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 35885 } }
I20260808 22:30:42.177549 1403 raft_consensus.cc:388] T 00000000000000000000000000000000 P 67efcefbc91e45dfae35a4db326f5ff8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:30:42.177630 1403 raft_consensus.cc:482] T 00000000000000000000000000000000 P 67efcefbc91e45dfae35a4db326f5ff8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:30:42.177744 1403 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 67efcefbc91e45dfae35a4db326f5ff8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:30:42.178557 1403 raft_consensus.cc:504] T 00000000000000000000000000000000 P 67efcefbc91e45dfae35a4db326f5ff8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67efcefbc91e45dfae35a4db326f5ff8" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 35885 } }
I20260808 22:30:42.178879 1403 leader_election.cc:302] T 00000000000000000000000000000000 P 67efcefbc91e45dfae35a4db326f5ff8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 67efcefbc91e45dfae35a4db326f5ff8; no voters:
I20260808 22:30:42.179172 1403 leader_election.cc:288] T 00000000000000000000000000000000 P 67efcefbc91e45dfae35a4db326f5ff8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:30:42.179255 1414 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 67efcefbc91e45dfae35a4db326f5ff8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:30:42.179510 1414 raft_consensus.cc:686] T 00000000000000000000000000000000 P 67efcefbc91e45dfae35a4db326f5ff8 [term 1 LEADER]: Becoming Leader. State: Replica: 67efcefbc91e45dfae35a4db326f5ff8, State: Running, Role: LEADER
I20260808 22:30:42.179833 1414 consensus_queue.cc:237] T 00000000000000000000000000000000 P 67efcefbc91e45dfae35a4db326f5ff8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67efcefbc91e45dfae35a4db326f5ff8" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 35885 } }
I20260808 22:30:42.180365 1403 sys_catalog.cc:564] T 00000000000000000000000000000000 P 67efcefbc91e45dfae35a4db326f5ff8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:30:42.181634 1416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67efcefbc91e45dfae35a4db326f5ff8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "67efcefbc91e45dfae35a4db326f5ff8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67efcefbc91e45dfae35a4db326f5ff8" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 35885 } } }
I20260808 22:30:42.181777 1416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67efcefbc91e45dfae35a4db326f5ff8 [sys.catalog]: This master's current role is: LEADER
I20260808 22:30:42.182018 1420 sys_catalog.cc:455] T 00000000000000000000000000000000 P 67efcefbc91e45dfae35a4db326f5ff8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 67efcefbc91e45dfae35a4db326f5ff8. Latest consensus state: current_term: 1 leader_uuid: "67efcefbc91e45dfae35a4db326f5ff8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "67efcefbc91e45dfae35a4db326f5ff8" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 35885 } } }
I20260808 22:30:42.182124 1420 sys_catalog.cc:458] T 00000000000000000000000000000000 P 67efcefbc91e45dfae35a4db326f5ff8 [sys.catalog]: This master's current role is: LEADER
I20260808 22:30:42.182775 1434 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:30:42.185534 1434 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:30:42.190433 1434 catalog_manager.cc:1347] Generated new cluster ID: f7c2fb2319dc44fe80a4f2e30209596f
I20260808 22:30:42.190495 1434 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:30:42.200199 1434 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:30:42.201330 1434 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:30:42.209127 1434 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 67efcefbc91e45dfae35a4db326f5ff8: Generated new TSK 0
I20260808 22:30:42.209738 1434 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:30:42.216269 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.193:0
--local_ip_for_outbound_sockets=127.5.11.193
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:35885
--builtin_ntp_servers=127.5.11.212:39859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 22:30:42.329002 1447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:30:42.329249 1447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:30:42.329324 1447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:30:42.333348 1447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:30:42.333396 1447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:30:42.333472 1447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.193
I20260808 22:30:42.338281 1447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:39859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:35885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.193
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:30:42.339330 1447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:30:42.340731 1447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:30:42.350170 1458 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:42.350171 1463 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:42.350149 1457 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:42.350414 1447 server_base.cc:1034] running on GCE node
I20260808 22:30:42.350850 1447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:30:42.351402 1447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:30:42.352567 1447 hybrid_clock.cc:648] HybridClock initialized: now 1786228242352546 us; error 30 us; skew 500 ppm
I20260808 22:30:42.354383 1447 webserver.cc:460] Webserver started at http://127.5.11.193:33227/ using document root <none> and password file <none>
I20260808 22:30:42.354909 1447 fs_manager.cc:359] Metadata directory not provided
I20260808 22:30:42.354969 1447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:30:42.355134 1447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:30:42.357152 1447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-0/data/instance:
uuid: "67f66cbb849645888c06d6f76b5a4af4"
format_stamp: "Formatted at 2026-08-08 22:30:42 on dist-test-slave-7f01"
server_key: "c67ef599e1a953a5db16fa8a2f80a2f3"
server_key_iv: "650071f65928a2832d7917fa1c9bea1f"
server_key_version: "encryptionkey@0"
I20260808 22:30:42.357669 1447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance:
uuid: "67f66cbb849645888c06d6f76b5a4af4"
format_stamp: "Formatted at 2026-08-08 22:30:42 on dist-test-slave-7f01"
server_key: "c67ef599e1a953a5db16fa8a2f80a2f3"
server_key_iv: "650071f65928a2832d7917fa1c9bea1f"
server_key_version: "encryptionkey@0"
I20260808 22:30:42.361156 1447 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:30:42.363418 1475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:42.364625 1447 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:30:42.364766 1447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-0/wal
uuid: "67f66cbb849645888c06d6f76b5a4af4"
format_stamp: "Formatted at 2026-08-08 22:30:42 on dist-test-slave-7f01"
server_key: "c67ef599e1a953a5db16fa8a2f80a2f3"
server_key_iv: "650071f65928a2832d7917fa1c9bea1f"
server_key_version: "encryptionkey@0"
I20260808 22:30:42.364889 1447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:30:42.409606 1447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:30:42.410313 1447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:30:42.410530 1447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:30:42.411135 1447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:30:42.412148 1447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:30:42.412220 1447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:42.412292 1447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:30:42.412331 1447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:42.421387 1447 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.193:35365
I20260808 22:30:42.421416 1647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.193:35365 every 8 connection(s)
I20260808 22:30:42.422263 1447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
I20260808 22:30:42.423849 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 1447
I20260808 22:30:42.423990 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance
I20260808 22:30:42.424331 5167 external_mini_cluster.cc:1428] Setting key ec54dfb3cb83798ff13cd0a005aa88d9
I20260808 22:30:42.426263 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.194:0
--local_ip_for_outbound_sockets=127.5.11.194
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:35885
--builtin_ntp_servers=127.5.11.212:39859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 22:30:42.431814 1412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:42.424133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:46855 (local address 127.5.11.254:35885)
0808 22:30:42.424265 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:42.424270 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:42.425551 (+ 1281us) server_negotiation.cc:407] Connection header received
0808 22:30:42.427073 (+ 1522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:42.427076 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:42.427162 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:42.427258 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:42.428198 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:42.428726 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:42.429854 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:42.430073 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:42.430177 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:42.431267 (+ 1090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:42.431271 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:42.431273 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:30:42.431292 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:42.431312 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:30:42.431320 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:42.431402 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:42.431563 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 22:30:42.431639 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 22:30:42.432579 1652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:42.424674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:35885 (local address 127.5.11.193:46855)
0808 22:30:42.425399 (+ 725us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:42.425451 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:42.426871 (+ 1420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:42.427434 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:42.427440 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:42.427523 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:30:42.428060 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:42.428074 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:42.428861 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:42.428864 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:42.429722 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:42.429731 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:42.430231 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:42.430235 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:42.430399 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:42.431013 (+ 614us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:30:42.431014 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:30:42.431054 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:30:42.431114 (+ 60us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:30:42.431404 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:30:42.431417 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:30:42.431425 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 22:30:42.431548 (+ 123us) client_negotiation.cc:240] Negotiation successful
0808 22:30:42.431833 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":143,"threads_started":1}
I20260808 22:30:42.433903 1650 heartbeater.cc:347] Connected to a master server at 127.5.11.254:35885
I20260808 22:30:42.434134 1650 heartbeater.cc:464] Registering TS with master...
I20260808 22:30:42.434625 1650 heartbeater.cc:511] Master 127.5.11.254:35885 requested a full tablet report, sending...
I20260808 22:30:42.436250 1350 ts_manager.cc:194] Registered new tserver with Master: 67f66cbb849645888c06d6f76b5a4af4 (127.5.11.193:35365)
I20260808 22:30:42.437481 1350 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.11.193:46855
W20260808 22:30:42.541168 1653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:30:42.541440 1653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:30:42.541503 1653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:30:42.545491 1653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:30:42.545559 1653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:30:42.545640 1653 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.194
I20260808 22:30:42.550457 1653 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:39859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:35885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.194
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:30:42.551458 1653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:30:42.552938 1653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:30:42.562417 1667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:42.562451 1664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:42.562418 1665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:42.562727 1653 server_base.cc:1034] running on GCE node
I20260808 22:30:42.563143 1653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:30:42.563884 1653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:30:42.565047 1653 hybrid_clock.cc:648] HybridClock initialized: now 1786228242565028 us; error 29 us; skew 500 ppm
I20260808 22:30:42.566896 1653 webserver.cc:460] Webserver started at http://127.5.11.194:45749/ using document root <none> and password file <none>
I20260808 22:30:42.567505 1653 fs_manager.cc:359] Metadata directory not provided
I20260808 22:30:42.567588 1653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:30:42.567889 1653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:30:42.570030 1653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-1/data/instance:
uuid: "5fb82c3309d64ae78bcf555933026dbe"
format_stamp: "Formatted at 2026-08-08 22:30:42 on dist-test-slave-7f01"
server_key: "03ac8b3df4f741809d0c72f185f0f32e"
server_key_iv: "93ae39b18d9854a9b043226bf4a69c08"
server_key_version: "encryptionkey@0"
I20260808 22:30:42.570611 1653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance:
uuid: "5fb82c3309d64ae78bcf555933026dbe"
format_stamp: "Formatted at 2026-08-08 22:30:42 on dist-test-slave-7f01"
server_key: "03ac8b3df4f741809d0c72f185f0f32e"
server_key_iv: "93ae39b18d9854a9b043226bf4a69c08"
server_key_version: "encryptionkey@0"
I20260808 22:30:42.574563 1653 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260808 22:30:42.576996 1676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:42.578217 1653 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:30:42.578351 1653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-1/wal
uuid: "5fb82c3309d64ae78bcf555933026dbe"
format_stamp: "Formatted at 2026-08-08 22:30:42 on dist-test-slave-7f01"
server_key: "03ac8b3df4f741809d0c72f185f0f32e"
server_key_iv: "93ae39b18d9854a9b043226bf4a69c08"
server_key_version: "encryptionkey@0"
I20260808 22:30:42.578469 1653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:30:42.601979 1653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:30:42.602679 1653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:30:42.602875 1653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:30:42.603534 1653 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:30:42.604563 1653 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:30:42.604635 1653 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:42.604710 1653 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:30:42.604750 1653 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:42.613714 1653 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.194:40463
I20260808 22:30:42.613765 1848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.194:40463 every 8 connection(s)
I20260808 22:30:42.614620 1653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
I20260808 22:30:42.623265 1412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:42.616401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:45543 (local address 127.5.11.254:35885)
0808 22:30:42.616552 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:42.616556 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:42.617516 (+ 960us) server_negotiation.cc:407] Connection header received
0808 22:30:42.618884 (+ 1368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:42.618887 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:42.618936 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:42.619007 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:42.619931 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:42.620467 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:42.621537 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:42.621743 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:42.621817 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:42.622862 (+ 1045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:42.622865 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:42.622867 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:30:42.622880 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:42.622901 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:30:42.622910 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:42.622968 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:42.623100 (+ 132us) server_negotiation.cc:299] Negotiation successful
0808 22:30:42.623149 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 22:30:42.623731 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 1653
I20260808 22:30:42.623884 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance
I20260808 22:30:42.624192 5167 external_mini_cluster.cc:1428] Setting key 2986a117dedd6baab72658dbafdad904
I20260808 22:30:42.624245 1854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:42.616745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:35885 (local address 127.5.11.194:45543)
0808 22:30:42.617378 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:42.617418 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:42.618699 (+ 1281us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:42.619145 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:42.619152 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:42.619232 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:30:42.619803 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:42.619817 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:42.620573 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:42.620575 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:42.621424 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:42.621433 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:42.621869 (+ 436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:42.621871 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:42.622006 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:42.622620 (+ 614us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:30:42.622622 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:30:42.622661 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:30:42.622724 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:30:42.622981 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:30:42.622987 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:30:42.622992 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:30:42.623083 (+ 91us) client_negotiation.cc:240] Negotiation successful
0808 22:30:42.623291 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":113,"threads_started":1}
I20260808 22:30:42.625439 1849 heartbeater.cc:347] Connected to a master server at 127.5.11.254:35885
I20260808 22:30:42.625684 1849 heartbeater.cc:464] Registering TS with master...
I20260808 22:30:42.626212 1849 heartbeater.cc:511] Master 127.5.11.254:35885 requested a full tablet report, sending...
I20260808 22:30:42.626230 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.195:0
--local_ip_for_outbound_sockets=127.5.11.195
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:35885
--builtin_ntp_servers=127.5.11.212:39859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 22:30:42.627255 1347 ts_manager.cc:194] Registered new tserver with Master: 5fb82c3309d64ae78bcf555933026dbe (127.5.11.194:40463)
I20260808 22:30:42.627872 1347 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.11.194:45543
W20260808 22:30:42.742468 1856 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:30:42.742712 1856 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:30:42.742769 1856 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:30:42.746657 1856 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:30:42.746703 1856 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:30:42.746812 1856 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.195
I20260808 22:30:42.751376 1856 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:39859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:35885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1856
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.195
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:30:42.752388 1856 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:30:42.753695 1856 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:30:42.762831 1866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:42.762859 1865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:42.762912 1856 server_base.cc:1034] running on GCE node
W20260808 22:30:42.762859 1871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:42.763489 1856 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:30:42.764129 1856 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:30:42.765288 1856 hybrid_clock.cc:648] HybridClock initialized: now 1786228242765275 us; error 23 us; skew 500 ppm
I20260808 22:30:42.767081 1856 webserver.cc:460] Webserver started at http://127.5.11.195:41057/ using document root <none> and password file <none>
I20260808 22:30:42.767648 1856 fs_manager.cc:359] Metadata directory not provided
I20260808 22:30:42.767712 1856 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:30:42.768077 1856 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:30:42.770287 1856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-2/data/instance:
uuid: "323412a02f6b42dd9880dae6304ebffa"
format_stamp: "Formatted at 2026-08-08 22:30:42 on dist-test-slave-7f01"
server_key: "a4d2f872742c6da4b10a9fb4c5dc362c"
server_key_iv: "f13a07640227dcaefa02090a6a8485d3"
server_key_version: "encryptionkey@0"
I20260808 22:30:42.770866 1856 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance:
uuid: "323412a02f6b42dd9880dae6304ebffa"
format_stamp: "Formatted at 2026-08-08 22:30:42 on dist-test-slave-7f01"
server_key: "a4d2f872742c6da4b10a9fb4c5dc362c"
server_key_iv: "f13a07640227dcaefa02090a6a8485d3"
server_key_version: "encryptionkey@0"
I20260808 22:30:42.774557 1856 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 22:30:42.776906 1879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:42.778086 1856 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:30:42.778216 1856 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-2/wal
uuid: "323412a02f6b42dd9880dae6304ebffa"
format_stamp: "Formatted at 2026-08-08 22:30:42 on dist-test-slave-7f01"
server_key: "a4d2f872742c6da4b10a9fb4c5dc362c"
server_key_iv: "f13a07640227dcaefa02090a6a8485d3"
server_key_version: "encryptionkey@0"
I20260808 22:30:42.778321 1856 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:30:42.794358 1856 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:30:42.795012 1856 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:30:42.795208 1856 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:30:42.795801 1856 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:30:42.796733 1856 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:30:42.796783 1856 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:42.796847 1856 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:30:42.796880 1856 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:42.805363 1856 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.195:40759
I20260808 22:30:42.805430 2047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.195:40759 every 8 connection(s)
I20260808 22:30:42.806265 1856 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
I20260808 22:30:42.814129 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 1856
I20260808 22:30:42.814267 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance
I20260808 22:30:42.814556 5167 external_mini_cluster.cc:1428] Setting key 8ef8d2585e06478e9b20b59eeff61c06
I20260808 22:30:42.815568 1412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:42.808119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:37767 (local address 127.5.11.254:35885)
0808 22:30:42.808255 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:42.808259 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:42.809325 (+ 1066us) server_negotiation.cc:407] Connection header received
0808 22:30:42.810745 (+ 1420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:42.810748 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:42.810808 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:42.810888 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:42.811768 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:42.812307 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:42.813447 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:42.813680 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:42.813789 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:42.815006 (+ 1217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:42.815010 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:42.815013 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:30:42.815048 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:42.815075 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:30:42.815087 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:42.815175 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:42.815351 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 22:30:42.815427 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260808 22:30:42.816516 2051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:42.808550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:35885 (local address 127.5.11.195:37767)
0808 22:30:42.809183 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:42.809224 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:42.810535 (+ 1311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:42.811057 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:42.811064 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:42.811144 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:30:42.811610 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:42.811622 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:42.812433 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:42.812438 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:42.813310 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:42.813320 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:42.813829 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:42.813833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:42.814013 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:42.814686 (+ 673us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:30:42.814688 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:30:42.814734 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:30:42.814797 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:30:42.815188 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:30:42.815196 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:30:42.815206 (+ 10us) client_negotiation.cc:769] Sending connection context
0808 22:30:42.815337 (+ 131us) client_negotiation.cc:240] Negotiation successful
0808 22:30:42.815613 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":114,"threads_started":1}
I20260808 22:30:42.817642 2049 heartbeater.cc:347] Connected to a master server at 127.5.11.254:35885
I20260808 22:30:42.817868 2049 heartbeater.cc:464] Registering TS with master...
I20260808 22:30:42.818385 2049 heartbeater.cc:511] Master 127.5.11.254:35885 requested a full tablet report, sending...
I20260808 22:30:42.819368 1347 ts_manager.cc:194] Registered new tserver with Master: 323412a02f6b42dd9880dae6304ebffa (127.5.11.195:40759)
I20260808 22:30:42.820005 1347 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.11.195:37767
I20260808 22:30:42.828370 5167 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:30:42.833679 1412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:42.830301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37990 (local address 127.5.11.254:35885)
0808 22:30:42.830423 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:42.830426 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:42.830631 (+ 205us) server_negotiation.cc:407] Connection header received
0808 22:30:42.830724 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:42.830726 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:42.830762 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:42.830828 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:42.831071 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:42.831554 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:42.832375 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:42.832586 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:42.832661 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:42.833366 (+ 705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:42.833369 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:42.833371 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:30:42.833382 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:42.833401 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:30:42.833409 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:42.833475 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:42.833537 (+ 62us) server_negotiation.cc:299] Negotiation successful
0808 22:30:42.833581 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260808 22:30:42.836477 1347 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37990:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:30:42.840116 1347 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:30:42.854413 2064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:42.848432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:35365 (local address 127.0.0.1:35490)
0808 22:30:42.849182 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:42.849199 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:42.849289 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:42.850187 (+ 898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:42.850193 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:42.850224 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:30:42.850427 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:42.850435 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:42.851374 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:42.851378 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:42.852808 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:42.852819 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:42.853890 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:42.853895 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:42.854001 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:42.854028 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 22:30:42.854088 (+ 60us) client_negotiation.cc:240] Negotiation successful
0808 22:30:42.854246 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":682,"mutex_wait_us":118,"spinlock_wait_cycles":3328,"thread_start_us":108,"threads_started":1}
I20260808 22:30:42.854784 2063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:42.848357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54734 (local address 127.5.11.195:40759)
0808 22:30:42.849208 (+ 851us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:42.849221 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:42.849248 (+ 27us) server_negotiation.cc:407] Connection header received
0808 22:30:42.849647 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:42.849655 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:42.849886 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:42.850060 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:30:42.850554 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:42.851707 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:42.853157 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:42.853720 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:42.853842 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:42.854037 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:42.854519 (+ 482us) server_negotiation.cc:299] Negotiation successful
0808 22:30:42.854628 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":729,"thread_start_us":155,"threads_started":1}
I20260808 22:30:42.854821 2067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:42.849123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:40463 (local address 127.0.0.1:35416)
0808 22:30:42.849529 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:42.849543 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:42.849622 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:42.850551 (+ 929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:42.850553 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:42.850572 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:30:42.850650 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:42.850655 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:42.852073 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:42.852079 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:42.853161 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:42.853172 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:42.854146 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:42.854149 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:42.854241 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:42.854262 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 22:30:42.854324 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 22:30:42.854378 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":97,"threads_started":1}
I20260808 22:30:42.855013 2066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:42.849032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35490 (local address 127.5.11.193:35365)
0808 22:30:42.849634 (+ 602us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:42.849647 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:42.849674 (+ 27us) server_negotiation.cc:407] Connection header received
0808 22:30:42.849788 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:42.849793 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:42.850002 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:42.850377 (+ 375us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:30:42.850533 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:42.851259 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:42.852952 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:42.853747 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:42.854492 (+ 745us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:42.854677 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:42.854738 (+ 61us) server_negotiation.cc:299] Negotiation successful
0808 22:30:42.854828 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":121,"threads_started":1}
I20260808 22:30:42.854417 2062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:42.848216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:40759 (local address 127.0.0.1:54734)
0808 22:30:42.848570 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:42.848624 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:42.848899 (+ 275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:42.850223 (+ 1324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:42.850226 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:42.850245 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:30:42.850415 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:42.850423 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:42.851835 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:42.851838 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:42.853037 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:42.853046 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:42.853843 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:42.853847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:42.853963 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:42.854003 (+ 40us) client_negotiation.cc:769] Sending connection context
0808 22:30:42.854128 (+ 125us) client_negotiation.cc:240] Negotiation successful
0808 22:30:42.854251 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":153,"thread_start_us":72,"threads_started":1}
I20260808 22:30:42.855875 2069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:42.849553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35416 (local address 127.5.11.194:40463)
0808 22:30:42.850099 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:42.850111 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:42.850132 (+ 21us) server_negotiation.cc:407] Connection header received
0808 22:30:42.850209 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:42.850214 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:42.850386 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:42.850790 (+ 404us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:30:42.850925 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:42.851969 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:42.853283 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:42.854030 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:42.854133 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:42.854314 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:42.854366 (+ 52us) server_negotiation.cc:299] Negotiation successful
0808 22:30:42.854477 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":120,"threads_started":1}
I20260808 22:30:42.857700 1943 tablet_service.cc:1467] Processing CreateTablet for tablet f8c62b57f67c41b9bed85f06fb6f6d47 (DEFAULT_TABLE table=test-table [id=ecf58e41a5824de4a1332a90f6322575]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:30:42.857851 1551 tablet_service.cc:1467] Processing CreateTablet for tablet f8c62b57f67c41b9bed85f06fb6f6d47 (DEFAULT_TABLE table=test-table [id=ecf58e41a5824de4a1332a90f6322575]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:30:42.858601 1943 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8c62b57f67c41b9bed85f06fb6f6d47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:30:42.858335 1743 tablet_service.cc:1467] Processing CreateTablet for tablet f8c62b57f67c41b9bed85f06fb6f6d47 (DEFAULT_TABLE table=test-table [id=ecf58e41a5824de4a1332a90f6322575]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:30:42.858871 1551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8c62b57f67c41b9bed85f06fb6f6d47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:30:42.859236 1743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f8c62b57f67c41b9bed85f06fb6f6d47. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:30:42.864758 2070 tablet_bootstrap.cc:492] T f8c62b57f67c41b9bed85f06fb6f6d47 P 323412a02f6b42dd9880dae6304ebffa: Bootstrap starting.
I20260808 22:30:42.866235 2073 tablet_bootstrap.cc:492] T f8c62b57f67c41b9bed85f06fb6f6d47 P 67f66cbb849645888c06d6f76b5a4af4: Bootstrap starting.
I20260808 22:30:42.867584 2070 tablet_bootstrap.cc:654] T f8c62b57f67c41b9bed85f06fb6f6d47 P 323412a02f6b42dd9880dae6304ebffa: Neither blocks nor log segments found. Creating new log.
I20260808 22:30:42.868108 2075 tablet_bootstrap.cc:492] T f8c62b57f67c41b9bed85f06fb6f6d47 P 5fb82c3309d64ae78bcf555933026dbe: Bootstrap starting.
I20260808 22:30:42.868391 2073 tablet_bootstrap.cc:654] T f8c62b57f67c41b9bed85f06fb6f6d47 P 67f66cbb849645888c06d6f76b5a4af4: Neither blocks nor log segments found. Creating new log.
I20260808 22:30:42.868563 2070 log.cc:824] T f8c62b57f67c41b9bed85f06fb6f6d47 P 323412a02f6b42dd9880dae6304ebffa: Log is configured to *not* fsync() on all Append() calls
I20260808 22:30:42.869159 2073 log.cc:824] T f8c62b57f67c41b9bed85f06fb6f6d47 P 67f66cbb849645888c06d6f76b5a4af4: Log is configured to *not* fsync() on all Append() calls
I20260808 22:30:42.870393 2075 tablet_bootstrap.cc:654] T f8c62b57f67c41b9bed85f06fb6f6d47 P 5fb82c3309d64ae78bcf555933026dbe: Neither blocks nor log segments found. Creating new log.
I20260808 22:30:42.870704 2070 tablet_bootstrap.cc:492] T f8c62b57f67c41b9bed85f06fb6f6d47 P 323412a02f6b42dd9880dae6304ebffa: No bootstrap required, opened a new log
I20260808 22:30:42.870901 2070 ts_tablet_manager.cc:1397] T f8c62b57f67c41b9bed85f06fb6f6d47 P 323412a02f6b42dd9880dae6304ebffa: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 22:30:42.871097 2073 tablet_bootstrap.cc:492] T f8c62b57f67c41b9bed85f06fb6f6d47 P 67f66cbb849645888c06d6f76b5a4af4: No bootstrap required, opened a new log
I20260808 22:30:42.871140 2075 log.cc:824] T f8c62b57f67c41b9bed85f06fb6f6d47 P 5fb82c3309d64ae78bcf555933026dbe: Log is configured to *not* fsync() on all Append() calls
I20260808 22:30:42.871305 2073 ts_tablet_manager.cc:1397] T f8c62b57f67c41b9bed85f06fb6f6d47 P 67f66cbb849645888c06d6f76b5a4af4: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 22:30:42.873296 2075 tablet_bootstrap.cc:492] T f8c62b57f67c41b9bed85f06fb6f6d47 P 5fb82c3309d64ae78bcf555933026dbe: No bootstrap required, opened a new log
I20260808 22:30:42.873494 2075 ts_tablet_manager.cc:1397] T f8c62b57f67c41b9bed85f06fb6f6d47 P 5fb82c3309d64ae78bcf555933026dbe: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 22:30:42.875311 2070 raft_consensus.cc:348] T f8c62b57f67c41b9bed85f06fb6f6d47 P 323412a02f6b42dd9880dae6304ebffa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "323412a02f6b42dd9880dae6304ebffa" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 40759 } } peers { permanent_uuid: "67f66cbb849645888c06d6f76b5a4af4" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 35365 } } peers { permanent_uuid: "5fb82c3309d64ae78bcf555933026dbe" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 40463 } }
I20260808 22:30:42.875622 2070 raft_consensus.cc:374] T f8c62b57f67c41b9bed85f06fb6f6d47 P 323412a02f6b42dd9880dae6304ebffa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:30:42.875686 2070 raft_consensus.cc:729] T f8c62b57f67c41b9bed85f06fb6f6d47 P 323412a02f6b42dd9880dae6304ebffa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 323412a02f6b42dd9880dae6304ebffa, State: Initialized, Role: FOLLOWER
I20260808 22:30:42.875610 2073 raft_consensus.cc:348] T f8c62b57f67c41b9bed85f06fb6f6d47 P 67f66cbb849645888c06d6f76b5a4af4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "323412a02f6b42dd9880dae6304ebffa" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 40759 } } peers { permanent_uuid: "67f66cbb849645888c06d6f76b5a4af4" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 35365 } } peers { permanent_uuid: "5fb82c3309d64ae78bcf555933026dbe" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 40463 } }
I20260808 22:30:42.875883 2073 raft_consensus.cc:374] T f8c62b57f67c41b9bed85f06fb6f6d47 P 67f66cbb849645888c06d6f76b5a4af4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:30:42.876024 2073 raft_consensus.cc:729] T f8c62b57f67c41b9bed85f06fb6f6d47 P 67f66cbb849645888c06d6f76b5a4af4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 67f66cbb849645888c06d6f76b5a4af4, State: Initialized, Role: FOLLOWER
I20260808 22:30:42.876178 2070 consensus_queue.cc:260] T f8c62b57f67c41b9bed85f06fb6f6d47 P 323412a02f6b42dd9880dae6304ebffa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "323412a02f6b42dd9880dae6304ebffa" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 40759 } } peers { permanent_uuid: "67f66cbb849645888c06d6f76b5a4af4" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 35365 } } peers { permanent_uuid: "5fb82c3309d64ae78bcf555933026dbe" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 40463 } }
I20260808 22:30:42.876550 2073 consensus_queue.cc:260] T f8c62b57f67c41b9bed85f06fb6f6d47 P 67f66cbb849645888c06d6f76b5a4af4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "323412a02f6b42dd9880dae6304ebffa" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 40759 } } peers { permanent_uuid: "67f66cbb849645888c06d6f76b5a4af4" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 35365 } } peers { permanent_uuid: "5fb82c3309d64ae78bcf555933026dbe" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 40463 } }
I20260808 22:30:42.876993 2049 heartbeater.cc:503] Master 127.5.11.254:35885 was elected leader, sending a full tablet report...
I20260808 22:30:42.877187 2070 ts_tablet_manager.cc:1428] T f8c62b57f67c41b9bed85f06fb6f6d47 P 323412a02f6b42dd9880dae6304ebffa: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 22:30:42.877074 2075 raft_consensus.cc:348] T f8c62b57f67c41b9bed85f06fb6f6d47 P 5fb82c3309d64ae78bcf555933026dbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "323412a02f6b42dd9880dae6304ebffa" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 40759 } } peers { permanent_uuid: "67f66cbb849645888c06d6f76b5a4af4" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 35365 } } peers { permanent_uuid: "5fb82c3309d64ae78bcf555933026dbe" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 40463 } }
I20260808 22:30:42.877329 2075 raft_consensus.cc:374] T f8c62b57f67c41b9bed85f06fb6f6d47 P 5fb82c3309d64ae78bcf555933026dbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:30:42.877388 2075 raft_consensus.cc:729] T f8c62b57f67c41b9bed85f06fb6f6d47 P 5fb82c3309d64ae78bcf555933026dbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fb82c3309d64ae78bcf555933026dbe, State: Initialized, Role: FOLLOWER
I20260808 22:30:42.877733 1650 heartbeater.cc:503] Master 127.5.11.254:35885 was elected leader, sending a full tablet report...
I20260808 22:30:42.877784 2075 consensus_queue.cc:260] T f8c62b57f67c41b9bed85f06fb6f6d47 P 5fb82c3309d64ae78bcf555933026dbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "323412a02f6b42dd9880dae6304ebffa" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 40759 } } peers { permanent_uuid: "67f66cbb849645888c06d6f76b5a4af4" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 35365 } } peers { permanent_uuid: "5fb82c3309d64ae78bcf555933026dbe" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 40463 } }
I20260808 22:30:42.878583 2073 ts_tablet_manager.cc:1428] T f8c62b57f67c41b9bed85f06fb6f6d47 P 67f66cbb849645888c06d6f76b5a4af4: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 22:30:42.878751 2075 ts_tablet_manager.cc:1428] T f8c62b57f67c41b9bed85f06fb6f6d47 P 5fb82c3309d64ae78bcf555933026dbe: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 22:30:42.879089 1849 heartbeater.cc:503] Master 127.5.11.254:35885 was elected leader, sending a full tablet report...
I20260808 22:30:42.899718 2082 raft_consensus.cc:482] T f8c62b57f67c41b9bed85f06fb6f6d47 P 67f66cbb849645888c06d6f76b5a4af4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:30:42.899888 2082 raft_consensus.cc:504] T f8c62b57f67c41b9bed85f06fb6f6d47 P 67f66cbb849645888c06d6f76b5a4af4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "323412a02f6b42dd9880dae6304ebffa" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 40759 } } peers { permanent_uuid: "67f66cbb849645888c06d6f76b5a4af4" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 35365 } } peers { permanent_uuid: "5fb82c3309d64ae78bcf555933026dbe" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 40463 } }
I20260808 22:30:42.900918 2082 leader_election.cc:288] T f8c62b57f67c41b9bed85f06fb6f6d47 P 67f66cbb849645888c06d6f76b5a4af4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 323412a02f6b42dd9880dae6304ebffa (127.5.11.195:40759), 5fb82c3309d64ae78bcf555933026dbe (127.5.11.194:40463)
I20260808 22:30:42.905578 2063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:42.901197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:48607 (local address 127.5.11.195:40759)
0808 22:30:42.901383 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:42.901389 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:42.901407 (+ 18us) server_negotiation.cc:407] Connection header received
0808 22:30:42.901495 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:42.901497 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:42.901536 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:42.901599 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:30:42.901962 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:42.902720 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:42.904357 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:42.904958 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:42.905078 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:42.905127 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:42.905382 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 22:30:42.905453 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260808 22:30:42.905578 2069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:42.901229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:33877 (local address 127.5.11.194:40463)
0808 22:30:42.901355 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:42.901361 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:42.901682 (+ 321us) server_negotiation.cc:407] Connection header received
0808 22:30:42.901770 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:42.901772 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:42.901810 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:42.901867 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:30:42.902142 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:42.902846 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:42.904357 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:42.904951 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:42.905078 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:42.905127 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:42.905382 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 22:30:42.905462 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 22:30:42.905596 1652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:42.901115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:40759 (local address 127.5.11.193:48607)
0808 22:30:42.901290 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:42.901307 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:42.901393 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:42.901641 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:42.901646 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:42.901666 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:30:42.901815 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:42.901822 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:42.902825 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:42.902827 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:42.904223 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:42.904235 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:42.905077 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:42.905081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:42.905237 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:42.905279 (+ 42us) client_negotiation.cc:769] Sending connection context
0808 22:30:42.905353 (+ 74us) client_negotiation.cc:240] Negotiation successful
0808 22:30:42.905444 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":123}
I20260808 22:30:42.905607 2084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:42.901166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:40463 (local address 127.5.11.193:33877)
0808 22:30:42.901561 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:42.901572 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:42.901683 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:42.901911 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:42.901914 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:42.901935 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:30:42.902036 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:42.902041 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:42.902942 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:42.902945 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:42.904223 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:42.904235 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:42.905077 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:42.905081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:42.905232 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:42.905276 (+ 44us) client_negotiation.cc:769] Sending connection context
0808 22:30:42.905353 (+ 77us) client_negotiation.cc:240] Negotiation successful
0808 22:30:42.905445 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":74,"spinlock_wait_cycles":1408,"thread_start_us":112,"threads_started":1}
I20260808 22:30:42.906270 1980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8c62b57f67c41b9bed85f06fb6f6d47" candidate_uuid: "67f66cbb849645888c06d6f76b5a4af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "323412a02f6b42dd9880dae6304ebffa" is_pre_election: true
I20260808 22:30:42.906467 1773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8c62b57f67c41b9bed85f06fb6f6d47" candidate_uuid: "67f66cbb849645888c06d6f76b5a4af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fb82c3309d64ae78bcf555933026dbe" is_pre_election: true
I20260808 22:30:42.906590 1980 raft_consensus.cc:2454] T f8c62b57f67c41b9bed85f06fb6f6d47 P 323412a02f6b42dd9880dae6304ebffa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67f66cbb849645888c06d6f76b5a4af4 in term 0.
I20260808 22:30:42.906720 1773 raft_consensus.cc:2454] T f8c62b57f67c41b9bed85f06fb6f6d47 P 5fb82c3309d64ae78bcf555933026dbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 67f66cbb849645888c06d6f76b5a4af4 in term 0.
I20260808 22:30:42.907148 1478 leader_election.cc:302] T f8c62b57f67c41b9bed85f06fb6f6d47 P 67f66cbb849645888c06d6f76b5a4af4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 323412a02f6b42dd9880dae6304ebffa, 67f66cbb849645888c06d6f76b5a4af4; no voters:
I20260808 22:30:42.907436 2082 raft_consensus.cc:2781] T f8c62b57f67c41b9bed85f06fb6f6d47 P 67f66cbb849645888c06d6f76b5a4af4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:30:42.907527 2082 raft_consensus.cc:482] T f8c62b57f67c41b9bed85f06fb6f6d47 P 67f66cbb849645888c06d6f76b5a4af4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:30:42.907573 2082 raft_consensus.cc:3037] T f8c62b57f67c41b9bed85f06fb6f6d47 P 67f66cbb849645888c06d6f76b5a4af4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:30:42.908679 2082 raft_consensus.cc:504] T f8c62b57f67c41b9bed85f06fb6f6d47 P 67f66cbb849645888c06d6f76b5a4af4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "323412a02f6b42dd9880dae6304ebffa" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 40759 } } peers { permanent_uuid: "67f66cbb849645888c06d6f76b5a4af4" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 35365 } } peers { permanent_uuid: "5fb82c3309d64ae78bcf555933026dbe" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 40463 } }
I20260808 22:30:42.909055 2082 leader_election.cc:288] T f8c62b57f67c41b9bed85f06fb6f6d47 P 67f66cbb849645888c06d6f76b5a4af4 [CANDIDATE]: Term 1 election: Requested vote from peers 323412a02f6b42dd9880dae6304ebffa (127.5.11.195:40759), 5fb82c3309d64ae78bcf555933026dbe (127.5.11.194:40463)
I20260808 22:30:42.909325 1980 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8c62b57f67c41b9bed85f06fb6f6d47" candidate_uuid: "67f66cbb849645888c06d6f76b5a4af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "323412a02f6b42dd9880dae6304ebffa"
I20260808 22:30:42.909411 1773 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f8c62b57f67c41b9bed85f06fb6f6d47" candidate_uuid: "67f66cbb849645888c06d6f76b5a4af4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fb82c3309d64ae78bcf555933026dbe"
I20260808 22:30:42.909456 1980 raft_consensus.cc:3037] T f8c62b57f67c41b9bed85f06fb6f6d47 P 323412a02f6b42dd9880dae6304ebffa [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:30:42.909540 1773 raft_consensus.cc:3037] T f8c62b57f67c41b9bed85f06fb6f6d47 P 5fb82c3309d64ae78bcf555933026dbe [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:30:42.910754 1980 raft_consensus.cc:2454] T f8c62b57f67c41b9bed85f06fb6f6d47 P 323412a02f6b42dd9880dae6304ebffa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67f66cbb849645888c06d6f76b5a4af4 in term 1.
I20260808 22:30:42.910754 1773 raft_consensus.cc:2454] T f8c62b57f67c41b9bed85f06fb6f6d47 P 5fb82c3309d64ae78bcf555933026dbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 67f66cbb849645888c06d6f76b5a4af4 in term 1.
I20260808 22:30:42.911067 1478 leader_election.cc:302] T f8c62b57f67c41b9bed85f06fb6f6d47 P 67f66cbb849645888c06d6f76b5a4af4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 323412a02f6b42dd9880dae6304ebffa, 67f66cbb849645888c06d6f76b5a4af4; no voters:
I20260808 22:30:42.911243 2082 raft_consensus.cc:2781] T f8c62b57f67c41b9bed85f06fb6f6d47 P 67f66cbb849645888c06d6f76b5a4af4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:30:42.911478 2082 raft_consensus.cc:686] T f8c62b57f67c41b9bed85f06fb6f6d47 P 67f66cbb849645888c06d6f76b5a4af4 [term 1 LEADER]: Becoming Leader. State: Replica: 67f66cbb849645888c06d6f76b5a4af4, State: Running, Role: LEADER
I20260808 22:30:42.911873 2082 consensus_queue.cc:237] T f8c62b57f67c41b9bed85f06fb6f6d47 P 67f66cbb849645888c06d6f76b5a4af4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "323412a02f6b42dd9880dae6304ebffa" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 40759 } } peers { permanent_uuid: "67f66cbb849645888c06d6f76b5a4af4" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 35365 } } peers { permanent_uuid: "5fb82c3309d64ae78bcf555933026dbe" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 40463 } }
I20260808 22:30:42.915093 1346 catalog_manager.cc:5495] T f8c62b57f67c41b9bed85f06fb6f6d47 P 67f66cbb849645888c06d6f76b5a4af4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 67f66cbb849645888c06d6f76b5a4af4 (127.5.11.193). New cstate: current_term: 1 leader_uuid: "67f66cbb849645888c06d6f76b5a4af4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "323412a02f6b42dd9880dae6304ebffa" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 40759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "67f66cbb849645888c06d6f76b5a4af4" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 35365 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5fb82c3309d64ae78bcf555933026dbe" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 40463 } health_report { overall_health: UNKNOWN } } }
W20260808 22:30:42.923367 1651 tablet.cc:2369] T f8c62b57f67c41b9bed85f06fb6f6d47 P 67f66cbb849645888c06d6f76b5a4af4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:30:42.935482 2066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:42.932040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35500 (local address 127.5.11.193:35365)
0808 22:30:42.932176 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:42.932181 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:42.932194 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:30:42.932347 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:42.932349 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:42.932402 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:42.932516 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:30:42.932752 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:42.933240 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:42.934081 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:42.934298 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:42.934400 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:42.934703 (+ 303us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:30:42.935087 (+ 384us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:30:42.935212 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:42.935271 (+ 59us) server_negotiation.cc:299] Negotiation successful
0808 22:30:42.935320 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 22:30:42.940975 1773 raft_consensus.cc:1261] T f8c62b57f67c41b9bed85f06fb6f6d47 P 5fb82c3309d64ae78bcf555933026dbe [term 1 FOLLOWER]: Refusing update from remote peer 67f66cbb849645888c06d6f76b5a4af4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:30:42.940979 1980 raft_consensus.cc:1261] T f8c62b57f67c41b9bed85f06fb6f6d47 P 323412a02f6b42dd9880dae6304ebffa [term 1 FOLLOWER]: Refusing update from remote peer 67f66cbb849645888c06d6f76b5a4af4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:30:42.941633 2082 consensus_queue.cc:1035] T f8c62b57f67c41b9bed85f06fb6f6d47 P 67f66cbb849645888c06d6f76b5a4af4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fb82c3309d64ae78bcf555933026dbe" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 40463 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:30:42.941798 2085 consensus_queue.cc:1035] T f8c62b57f67c41b9bed85f06fb6f6d47 P 67f66cbb849645888c06d6f76b5a4af4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "323412a02f6b42dd9880dae6304ebffa" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 40759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:30:42.949731 2094 mvcc.cc:205] Tried to move back new op lower bound from 7316390883079507968 to 7316390882969972736. Current Snapshot: MvccSnapshot[applied={T|T < 7316390883079507968}]
I20260808 22:30:42.951562 2092 mvcc.cc:205] Tried to move back new op lower bound from 7316390883079507968 to 7316390882969972736. Current Snapshot: MvccSnapshot[applied={T|T < 7316390883079507968}]
I20260808 22:30:42.956601 1347 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37990:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:30:42.956780 1347 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37990:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:30:42.959126 1347 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 67efcefbc91e45dfae35a4db326f5ff8: Sending DeleteTablet for 3 replicas of tablet f8c62b57f67c41b9bed85f06fb6f6d47
I20260808 22:30:42.959841 1943 tablet_service.cc:1514] Processing DeleteTablet for tablet f8c62b57f67c41b9bed85f06fb6f6d47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:30:42 UTC) from {username='slave'} at 127.0.0.1:54734
I20260808 22:30:42.960062 1551 tablet_service.cc:1514] Processing DeleteTablet for tablet f8c62b57f67c41b9bed85f06fb6f6d47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:30:42 UTC) from {username='slave'} at 127.0.0.1:35490
I20260808 22:30:42.960076 1743 tablet_service.cc:1514] Processing DeleteTablet for tablet f8c62b57f67c41b9bed85f06fb6f6d47 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:30:42 UTC) from {username='slave'} at 127.0.0.1:35416
I20260808 22:30:42.960331 2106 tablet_replica.cc:330] T f8c62b57f67c41b9bed85f06fb6f6d47 P 323412a02f6b42dd9880dae6304ebffa: stopping tablet replica
I20260808 22:30:42.960639 2106 raft_consensus.cc:2229] T f8c62b57f67c41b9bed85f06fb6f6d47 P 323412a02f6b42dd9880dae6304ebffa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:30:42.960810 2107 tablet_replica.cc:330] T f8c62b57f67c41b9bed85f06fb6f6d47 P 67f66cbb849645888c06d6f76b5a4af4: stopping tablet replica
I20260808 22:30:42.960975 2106 raft_consensus.cc:2258] T f8c62b57f67c41b9bed85f06fb6f6d47 P 323412a02f6b42dd9880dae6304ebffa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:30:42.961172 2107 raft_consensus.cc:2229] T f8c62b57f67c41b9bed85f06fb6f6d47 P 67f66cbb849645888c06d6f76b5a4af4 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:30:42.961349 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 1447
I20260808 22:30:42.965112 2106 ts_tablet_manager.cc:1905] T f8c62b57f67c41b9bed85f06fb6f6d47 P 323412a02f6b42dd9880dae6304ebffa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:30:42.969429 2106 ts_tablet_manager.cc:1918] T f8c62b57f67c41b9bed85f06fb6f6d47 P 323412a02f6b42dd9880dae6304ebffa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:30:42.969558 2106 log.cc:1196] T f8c62b57f67c41b9bed85f06fb6f6d47 P 323412a02f6b42dd9880dae6304ebffa: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-2/wal/wals/f8c62b57f67c41b9bed85f06fb6f6d47
I20260808 22:30:42.969674 2108 tablet_replica.cc:330] T f8c62b57f67c41b9bed85f06fb6f6d47 P 5fb82c3309d64ae78bcf555933026dbe: stopping tablet replica
I20260808 22:30:42.969885 2106 ts_tablet_manager.cc:1939] T f8c62b57f67c41b9bed85f06fb6f6d47 P 323412a02f6b42dd9880dae6304ebffa: Deleting consensus metadata
I20260808 22:30:42.970008 2108 raft_consensus.cc:2229] T f8c62b57f67c41b9bed85f06fb6f6d47 P 5fb82c3309d64ae78bcf555933026dbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:30:42.970326 2108 raft_consensus.cc:2258] T f8c62b57f67c41b9bed85f06fb6f6d47 P 5fb82c3309d64ae78bcf555933026dbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:30:42.971216 1318 catalog_manager.cc:4841] TS 323412a02f6b42dd9880dae6304ebffa (127.5.11.195:40759): tablet f8c62b57f67c41b9bed85f06fb6f6d47 (table test-table [id=ecf58e41a5824de4a1332a90f6322575]) successfully deleted
I20260808 22:30:42.971725 2108 ts_tablet_manager.cc:1905] T f8c62b57f67c41b9bed85f06fb6f6d47 P 5fb82c3309d64ae78bcf555933026dbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:30:42.973767 2108 ts_tablet_manager.cc:1918] T f8c62b57f67c41b9bed85f06fb6f6d47 P 5fb82c3309d64ae78bcf555933026dbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:30:42.973877 2108 log.cc:1196] T f8c62b57f67c41b9bed85f06fb6f6d47 P 5fb82c3309d64ae78bcf555933026dbe: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786227991988852-5167-0/minicluster-data/ts-1/wal/wals/f8c62b57f67c41b9bed85f06fb6f6d47
I20260808 22:30:42.974148 2108 ts_tablet_manager.cc:1939] T f8c62b57f67c41b9bed85f06fb6f6d47 P 5fb82c3309d64ae78bcf555933026dbe: Deleting consensus metadata
I20260808 22:30:42.975251 1320 catalog_manager.cc:4841] TS 5fb82c3309d64ae78bcf555933026dbe (127.5.11.194:40463): tablet f8c62b57f67c41b9bed85f06fb6f6d47 (table test-table [id=ecf58e41a5824de4a1332a90f6322575]) successfully deleted
W20260808 22:30:42.976697 1316 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.11.193:35365 (error 108)
I20260808 22:30:42.976994 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 1653
I20260808 22:30:42.979799 2062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:42.979428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:35365 (local address 127.0.0.1:35512)
0808 22:30:42.979592 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:42.979678 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.193:35365: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260808 22:30:42.982577 1316 catalog_manager.cc:4584] TS 67f66cbb849645888c06d6f76b5a4af4 (127.5.11.193:35365): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f8c62b57f67c41b9bed85f06fb6f6d47: Network error: Client connection negotiation failed: client connection to 127.5.11.193:35365: connect: Connection refused (error 111)
I20260808 22:30:42.988365 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 1856
I20260808 22:30:43.000981 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 1295
2026-08-08T22:30:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1052 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:30:43 dist-test-slave-7f01 krb5kdc[2113](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:30:43 dist-test-slave-7f01 krb5kdc[2113](info): set up 2 sockets
Aug 08 22:30:43 dist-test-slave-7f01 krb5kdc[2113](info): commencing operation
krb5kdc: starting...
W20260808 22:30:44.271078 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:30:44 dist-test-slave-7f01 krb5kdc[2113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:30:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:30:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.11.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
I20260808 22:30:44.399194 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:39537
--webserver_interface=127.5.11.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:37995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:39537
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:30:44.515492 2138 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:30:44.515805 2138 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:30:44.515921 2138 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:30:44.519860 2138 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:30:44.519930 2138 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:30:44.519968 2138 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:30:44.520002 2138 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:30:44.520036 2138 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:30:44.525133 2138 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:37995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:39537
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:39537
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2138
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:30:44.526113 2138 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:30:44.527109 2138 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:30:44.535303 2150 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:44.535359 2149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:44.535506 2138 server_base.cc:1034] running on GCE node
W20260808 22:30:44.535347 2153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:44.536230 2138 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:30:44.537251 2138 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:30:44.538455 2138 hybrid_clock.cc:648] HybridClock initialized: now 1786228244538428 us; error 32 us; skew 500 ppm
Aug 08 22:30:44 dist-test-slave-7f01 krb5kdc[2113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228244, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:30:44.543653 2138 init.cc:377] Logged in from keytab as kudu/127.5.11.254@KRBTEST.COM (short username kudu)
I20260808 22:30:44.544538 2138 webserver.cc:460] Webserver started at http://127.5.11.254:46737/ using document root <none> and password file <none>
I20260808 22:30:44.545092 2138 fs_manager.cc:359] Metadata directory not provided
I20260808 22:30:44.545186 2138 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:30:44.545471 2138 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:30:44.547495 2138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "2b76ea99d7464072a72107b1836615e8"
format_stamp: "Formatted at 2026-08-08 22:30:44 on dist-test-slave-7f01"
server_key: "a2e8476dfcc9dfddba70afdc567cc4e0"
server_key_iv: "464fa4b2d3de86c4ce4316d0b973b978"
server_key_version: "encryptionkey@0"
I20260808 22:30:44.548086 2138 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "2b76ea99d7464072a72107b1836615e8"
format_stamp: "Formatted at 2026-08-08 22:30:44 on dist-test-slave-7f01"
server_key: "a2e8476dfcc9dfddba70afdc567cc4e0"
server_key_iv: "464fa4b2d3de86c4ce4316d0b973b978"
server_key_version: "encryptionkey@0"
I20260808 22:30:44.551631 2138 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 22:30:44.553994 2165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:44.555107 2138 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:30:44.555251 2138 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "2b76ea99d7464072a72107b1836615e8"
format_stamp: "Formatted at 2026-08-08 22:30:44 on dist-test-slave-7f01"
server_key: "a2e8476dfcc9dfddba70afdc567cc4e0"
server_key_iv: "464fa4b2d3de86c4ce4316d0b973b978"
server_key_version: "encryptionkey@0"
I20260808 22:30:44.555364 2138 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:30:44.574121 2138 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:30:44.577165 2138 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:30:44.577371 2138 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:30:44.584899 2138 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:39537
I20260808 22:30:44.584928 2247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:39537 every 8 connection(s)
I20260808 22:30:44.585901 2138 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:30:44.586120 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 2138
I20260808 22:30:44.586282 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:30:44.586648 5167 external_mini_cluster.cc:1428] Setting key 88c26d47d6e3f5f7905a85f67c56eeca
I20260808 22:30:44.589278 2249 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 08 22:30:44 dist-test-slave-7f01 krb5kdc[2113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228244, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:30:44.596694 2249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b76ea99d7464072a72107b1836615e8: Bootstrap starting.
I20260808 22:30:44.599005 2249 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b76ea99d7464072a72107b1836615e8: Neither blocks nor log segments found. Creating new log.
I20260808 22:30:44.599939 2249 log.cc:824] T 00000000000000000000000000000000 P 2b76ea99d7464072a72107b1836615e8: Log is configured to *not* fsync() on all Append() calls
I20260808 22:30:44.601601 2254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:44.588151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34862 (local address 127.5.11.254:39537)
0808 22:30:44.588570 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:44.588588 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:44.588635 (+ 47us) server_negotiation.cc:407] Connection header received
0808 22:30:44.589274 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:44.589290 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:44.589678 (+ 388us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:44.589958 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:44.590979 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:44.592056 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:44.593305 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:44.593760 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:44.593876 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:44.596896 (+ 3020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:44.596902 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:44.596915 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:30:44.596956 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:44.598677 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:44.599154 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:44.599160 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:44.599164 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:44.599278 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:44.599575 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:44.599579 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:44.599581 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:44.600081 (+ 500us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:44.600298 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:44.600593 (+ 295us) server_negotiation.cc:299] Negotiation successful
0808 22:30:44.600888 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":151,"threads_started":1}
I20260808 22:30:44.602114 2249 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b76ea99d7464072a72107b1836615e8: No bootstrap required, opened a new log
I20260808 22:30:44.606418 2249 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b76ea99d7464072a72107b1836615e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b76ea99d7464072a72107b1836615e8" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39537 } }
I20260808 22:30:44.606684 2249 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b76ea99d7464072a72107b1836615e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:30:44.606786 2249 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b76ea99d7464072a72107b1836615e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b76ea99d7464072a72107b1836615e8, State: Initialized, Role: FOLLOWER
I20260808 22:30:44.607283 2249 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b76ea99d7464072a72107b1836615e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b76ea99d7464072a72107b1836615e8" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39537 } }
I20260808 22:30:44.607487 2249 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b76ea99d7464072a72107b1836615e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:30:44.607573 2249 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b76ea99d7464072a72107b1836615e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:30:44.607700 2249 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b76ea99d7464072a72107b1836615e8 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:30:44.608798 2249 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b76ea99d7464072a72107b1836615e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b76ea99d7464072a72107b1836615e8" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39537 } }
I20260808 22:30:44.609161 2249 leader_election.cc:302] T 00000000000000000000000000000000 P 2b76ea99d7464072a72107b1836615e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b76ea99d7464072a72107b1836615e8; no voters:
I20260808 22:30:44.609485 2249 leader_election.cc:288] T 00000000000000000000000000000000 P 2b76ea99d7464072a72107b1836615e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:30:44.609575 2260 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b76ea99d7464072a72107b1836615e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:30:44.609843 2260 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b76ea99d7464072a72107b1836615e8 [term 1 LEADER]: Becoming Leader. State: Replica: 2b76ea99d7464072a72107b1836615e8, State: Running, Role: LEADER
I20260808 22:30:44.610150 2260 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b76ea99d7464072a72107b1836615e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b76ea99d7464072a72107b1836615e8" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39537 } }
I20260808 22:30:44.610788 2249 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b76ea99d7464072a72107b1836615e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:30:44.612742 2262 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b76ea99d7464072a72107b1836615e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b76ea99d7464072a72107b1836615e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b76ea99d7464072a72107b1836615e8" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39537 } } }
I20260808 22:30:44.613011 2262 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b76ea99d7464072a72107b1836615e8 [sys.catalog]: This master's current role is: LEADER
I20260808 22:30:44.613046 2263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b76ea99d7464072a72107b1836615e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b76ea99d7464072a72107b1836615e8. Latest consensus state: current_term: 1 leader_uuid: "2b76ea99d7464072a72107b1836615e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b76ea99d7464072a72107b1836615e8" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39537 } } }
I20260808 22:30:44.613161 2263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b76ea99d7464072a72107b1836615e8 [sys.catalog]: This master's current role is: LEADER
W20260808 22:30:44.615540 2286 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2b76ea99d7464072a72107b1836615e8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:30:44.615607 2286 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:30:44.615689 2290 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:30:44.616422 2290 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:30:44.621726 2290 catalog_manager.cc:1347] Generated new cluster ID: e2518bdedf2c445598bc9c221c7e011c
I20260808 22:30:44.621802 2290 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:30:44.627354 2290 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:30:44.628237 2290 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:30:44.635422 2290 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b76ea99d7464072a72107b1836615e8: Generated new TSK 0
I20260808 22:30:44.636066 2290 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.11.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab.
Entry for principal kudu/127.5.11.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab.
I20260808 22:30:44.670536 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.193:0
--local_ip_for_outbound_sockets=127.5.11.193
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:39537
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:37995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:30:44.783955 2298 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:30:44.784192 2298 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:30:44.784233 2298 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:30:44.788267 2298 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:30:44.788318 2298 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:30:44.788409 2298 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.193
I20260808 22:30:44.793411 2298 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:37995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:39537
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2298
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.193
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:30:44.794412 2298 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:30:44.795419 2298 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:30:44.804979 2312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:44.804970 2311 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:44.805163 2298 server_base.cc:1034] running on GCE node
W20260808 22:30:44.805243 2314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:44.805836 2298 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:30:44.806620 2298 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:30:44.807830 2298 hybrid_clock.cc:648] HybridClock initialized: now 1786228244807791 us; error 49 us; skew 500 ppm
Aug 08 22:30:44 dist-test-slave-7f01 krb5kdc[2113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228244, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:30:44.810811 2298 init.cc:377] Logged in from keytab as kudu/127.5.11.193@KRBTEST.COM (short username kudu)
I20260808 22:30:44.811955 2298 webserver.cc:460] Webserver started at http://127.5.11.193:34685/ using document root <none> and password file <none>
I20260808 22:30:44.812582 2298 fs_manager.cc:359] Metadata directory not provided
I20260808 22:30:44.812649 2298 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:30:44.812872 2298 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:30:44.815037 2298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-0/data/instance:
uuid: "ce434990992f431e8caa51c180ee2693"
format_stamp: "Formatted at 2026-08-08 22:30:44 on dist-test-slave-7f01"
server_key: "b9b34fd8d790a26d335575e56596b705"
server_key_iv: "32ea2efb8de749345ea473b1827bb3dd"
server_key_version: "encryptionkey@0"
I20260808 22:30:44.815608 2298 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance:
uuid: "ce434990992f431e8caa51c180ee2693"
format_stamp: "Formatted at 2026-08-08 22:30:44 on dist-test-slave-7f01"
server_key: "b9b34fd8d790a26d335575e56596b705"
server_key_iv: "32ea2efb8de749345ea473b1827bb3dd"
server_key_version: "encryptionkey@0"
I20260808 22:30:44.819242 2298 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260808 22:30:44.821659 2332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:44.822832 2298 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:30:44.822983 2298 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-0/wal
uuid: "ce434990992f431e8caa51c180ee2693"
format_stamp: "Formatted at 2026-08-08 22:30:44 on dist-test-slave-7f01"
server_key: "b9b34fd8d790a26d335575e56596b705"
server_key_iv: "32ea2efb8de749345ea473b1827bb3dd"
server_key_version: "encryptionkey@0"
I20260808 22:30:44.823096 2298 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:30:44.840526 2298 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:30:44.843791 2298 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:30:44.843999 2298 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:30:44.844585 2298 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:30:44.845400 2298 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:30:44.845477 2298 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:44.845551 2298 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:30:44.845594 2298 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:44.854701 2298 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.193:43077
I20260808 22:30:44.854724 2529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.193:43077 every 8 connection(s)
I20260808 22:30:44.855619 2298 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
I20260808 22:30:44.857050 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 2298
I20260808 22:30:44.857193 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance
I20260808 22:30:44.857543 5167 external_mini_cluster.cc:1428] Setting key 939965f2fdba8847197f5fcf4fbc9d2f
Aug 08 22:30:44 dist-test-slave-7f01 krb5kdc[2113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228244, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.11.194@KRBTEST.COM; defaulting to no policy
I20260808 22:30:44.870949 2254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:44.857573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:52481 (local address 127.5.11.254:39537)
0808 22:30:44.857734 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:44.857737 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:44.858812 (+ 1075us) server_negotiation.cc:407] Connection header received
0808 22:30:44.860366 (+ 1554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:44.860372 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:44.860437 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:44.860526 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:44.861944 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:44.862484 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:44.863695 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:44.864019 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:44.864104 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:44.866996 (+ 2892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:44.867003 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:44.867008 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:30:44.867043 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:44.868775 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:44.869281 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:44.869285 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:44.869287 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:44.869376 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:44.869716 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:44.869720 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:44.869723 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:44.869988 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:44.870104 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:44.870597 (+ 493us) server_negotiation.cc:299] Negotiation successful
0808 22:30:44.870740 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260808 22:30:44.871847 2537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:44.858048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39537 (local address 127.5.11.193:52481)
0808 22:30:44.858647 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:44.858725 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:44.860163 (+ 1438us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:44.860686 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:44.860692 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:44.861039 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:30:44.861697 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:44.861719 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:44.862617 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:44.862623 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:44.863583 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:44.863594 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:44.864175 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:44.864182 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:44.864422 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:44.865059 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:30:44.865085 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:30:44.866821 (+ 1736us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:30:44.868901 (+ 2080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:30:44.868907 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:30:44.868918 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:30:44.869176 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:30:44.869481 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:30:44.869483 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:30:44.869484 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:30:44.869616 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:30:44.870107 (+ 491us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:30:44.870112 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:30:44.870338 (+ 226us) client_negotiation.cc:769] Sending connection context
0808 22:30:44.870499 (+ 161us) client_negotiation.cc:240] Negotiation successful
0808 22:30:44.870808 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":152,"threads_started":1}
I20260808 22:30:44.873128 2532 heartbeater.cc:347] Connected to a master server at 127.5.11.254:39537
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.194@KRBTEST.COM" created.
I20260808 22:30:44.873406 2532 heartbeater.cc:464] Registering TS with master...
I20260808 22:30:44.873907 2532 heartbeater.cc:511] Master 127.5.11.254:39537 requested a full tablet report, sending...
I20260808 22:30:44.875448 2187 ts_manager.cc:194] Registered new tserver with Master: ce434990992f431e8caa51c180ee2693 (127.5.11.193:43077)
I20260808 22:30:44.876914 2187 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.193@KRBTEST.COM'} at 127.5.11.193:52481
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab.
Entry for principal kudu/127.5.11.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab.
I20260808 22:30:44.890204 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.194:0
--local_ip_for_outbound_sockets=127.5.11.194
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:39537
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:37995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:30:45.003388 2544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:30:45.003649 2544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:30:45.003717 2544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:30:45.007644 2544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:30:45.007725 2544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:30:45.007828 2544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.194
I20260808 22:30:45.013113 2544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:37995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:39537
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.194
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:30:45.014074 2544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:30:45.015048 2544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:30:45.024469 2556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:45.024432 2550 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:45.024449 2544 server_base.cc:1034] running on GCE node
W20260808 22:30:45.024432 2554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:45.025108 2544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:30:45.025707 2544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:30:45.026926 2544 hybrid_clock.cc:648] HybridClock initialized: now 1786228245026908 us; error 38 us; skew 500 ppm
Aug 08 22:30:45 dist-test-slave-7f01 krb5kdc[2113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228245, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:30:45.029812 2544 init.cc:377] Logged in from keytab as kudu/127.5.11.194@KRBTEST.COM (short username kudu)
I20260808 22:30:45.030692 2544 webserver.cc:460] Webserver started at http://127.5.11.194:44775/ using document root <none> and password file <none>
I20260808 22:30:45.031240 2544 fs_manager.cc:359] Metadata directory not provided
I20260808 22:30:45.031343 2544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:30:45.031576 2544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:30:45.033608 2544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-1/data/instance:
uuid: "d62e7dd81270464d92448d7a1ab802da"
format_stamp: "Formatted at 2026-08-08 22:30:45 on dist-test-slave-7f01"
server_key: "c6985a20456758fb3111ffb51241890a"
server_key_iv: "fb25e1f75a8160815cc386417d24656f"
server_key_version: "encryptionkey@0"
I20260808 22:30:45.034155 2544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance:
uuid: "d62e7dd81270464d92448d7a1ab802da"
format_stamp: "Formatted at 2026-08-08 22:30:45 on dist-test-slave-7f01"
server_key: "c6985a20456758fb3111ffb51241890a"
server_key_iv: "fb25e1f75a8160815cc386417d24656f"
server_key_version: "encryptionkey@0"
I20260808 22:30:45.037853 2544 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260808 22:30:45.040360 2569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:45.041523 2544 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 22:30:45.041656 2544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-1/wal
uuid: "d62e7dd81270464d92448d7a1ab802da"
format_stamp: "Formatted at 2026-08-08 22:30:45 on dist-test-slave-7f01"
server_key: "c6985a20456758fb3111ffb51241890a"
server_key_iv: "fb25e1f75a8160815cc386417d24656f"
server_key_version: "encryptionkey@0"
I20260808 22:30:45.041764 2544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:30:45.066246 2544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:30:45.069662 2544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:30:45.069875 2544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:30:45.070487 2544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:30:45.071341 2544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:30:45.071413 2544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:45.071498 2544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:30:45.071552 2544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:45.080796 2544 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.194:36513
I20260808 22:30:45.080819 2774 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.194:36513 every 8 connection(s)
I20260808 22:30:45.081686 2544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
I20260808 22:30:45.086890 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 2544
I20260808 22:30:45.087029 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance
I20260808 22:30:45.087379 5167 external_mini_cluster.cc:1428] Setting key ecb2700a6f4d72d11b3bd59f386ba320
Aug 08 22:30:45 dist-test-slave-7f01 krb5kdc[2113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228245, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:30:45.096654 2254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:45.083515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:56629 (local address 127.5.11.254:39537)
0808 22:30:45.083669 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:45.083674 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:45.084573 (+ 899us) server_negotiation.cc:407] Connection header received
0808 22:30:45.085872 (+ 1299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:45.085875 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:45.085926 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:45.086007 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:45.087461 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:45.088202 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:45.089494 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:45.089722 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:45.089832 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:45.092728 (+ 2896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:45.092731 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:45.092733 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:30:45.092758 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:45.094281 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:45.094954 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:45.094957 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:45.094958 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:45.095037 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:45.095469 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:45.095471 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:45.095472 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:45.095640 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:45.095802 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:45.096435 (+ 633us) server_negotiation.cc:299] Negotiation successful
0808 22:30:45.096534 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 22:30:45.097857 2784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:45.083967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39537 (local address 127.5.11.194:56629)
0808 22:30:45.084418 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:45.084458 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:45.085651 (+ 1193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:45.086177 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:45.086184 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:45.086530 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:30:45.087277 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:45.087299 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:45.088334 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:45.088337 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:45.089358 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:45.089369 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:45.089893 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:45.089898 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:45.090065 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:45.090687 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:30:45.090703 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:30:45.092549 (+ 1846us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:30:45.094404 (+ 1855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:30:45.094412 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:30:45.094429 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:30:45.094841 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:30:45.095137 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:30:45.095142 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:30:45.095147 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:30:45.095365 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:30:45.095802 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:30:45.095811 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:30:45.096150 (+ 339us) client_negotiation.cc:769] Sending connection context
0808 22:30:45.096389 (+ 239us) client_negotiation.cc:240] Negotiation successful
0808 22:30:45.096747 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":119,"threads_started":1}
WARNING: no policy specified for kudu/127.5.11.195@KRBTEST.COM; defaulting to no policy
I20260808 22:30:45.099092 2777 heartbeater.cc:347] Connected to a master server at 127.5.11.254:39537
I20260808 22:30:45.099339 2777 heartbeater.cc:464] Registering TS with master...
I20260808 22:30:45.099933 2777 heartbeater.cc:511] Master 127.5.11.254:39537 requested a full tablet report, sending...
I20260808 22:30:45.101037 2187 ts_manager.cc:194] Registered new tserver with Master: d62e7dd81270464d92448d7a1ab802da (127.5.11.194:36513)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.195@KRBTEST.COM" created.
I20260808 22:30:45.101701 2187 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.194@KRBTEST.COM'} at 127.5.11.194:56629
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab.
Entry for principal kudu/127.5.11.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab.
I20260808 22:30:45.118891 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.195:0
--local_ip_for_outbound_sockets=127.5.11.195
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:39537
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:37995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:30:45.233037 2791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:30:45.233289 2791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:30:45.233330 2791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:30:45.237346 2791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:30:45.237402 2791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:30:45.237457 2791 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.195
I20260808 22:30:45.242519 2791 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:37995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:39537
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2791
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.195
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:30:45.243469 2791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:30:45.244534 2791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:30:45.254423 2799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:45.254415 2802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:45.254415 2800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:45.254832 2791 server_base.cc:1034] running on GCE node
I20260808 22:30:45.255211 2791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:30:45.255939 2791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:30:45.257145 2791 hybrid_clock.cc:648] HybridClock initialized: now 1786228245257115 us; error 40 us; skew 500 ppm
Aug 08 22:30:45 dist-test-slave-7f01 krb5kdc[2113](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228245, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:30:45.260129 2791 init.cc:377] Logged in from keytab as kudu/127.5.11.195@KRBTEST.COM (short username kudu)
I20260808 22:30:45.260986 2791 webserver.cc:460] Webserver started at http://127.5.11.195:46315/ using document root <none> and password file <none>
I20260808 22:30:45.261564 2791 fs_manager.cc:359] Metadata directory not provided
I20260808 22:30:45.261657 2791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:30:45.261900 2791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:30:45.263999 2791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-2/data/instance:
uuid: "eec9f8c31a3b4a30aa57be9bc56348c1"
format_stamp: "Formatted at 2026-08-08 22:30:45 on dist-test-slave-7f01"
server_key: "b9790d6d21106fb03982801b48bbd9ea"
server_key_iv: "054044d6d0fa3e5917316cbaea28cb9d"
server_key_version: "encryptionkey@0"
I20260808 22:30:45.264597 2791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance:
uuid: "eec9f8c31a3b4a30aa57be9bc56348c1"
format_stamp: "Formatted at 2026-08-08 22:30:45 on dist-test-slave-7f01"
server_key: "b9790d6d21106fb03982801b48bbd9ea"
server_key_iv: "054044d6d0fa3e5917316cbaea28cb9d"
server_key_version: "encryptionkey@0"
I20260808 22:30:45.268774 2791 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260808 22:30:45.271238 2814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:45.272531 2791 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:30:45.272670 2791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-2/wal
uuid: "eec9f8c31a3b4a30aa57be9bc56348c1"
format_stamp: "Formatted at 2026-08-08 22:30:45 on dist-test-slave-7f01"
server_key: "b9790d6d21106fb03982801b48bbd9ea"
server_key_iv: "054044d6d0fa3e5917316cbaea28cb9d"
server_key_version: "encryptionkey@0"
I20260808 22:30:45.272783 2791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:30:45.291177 2791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:30:45.294534 2791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:30:45.294749 2791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:30:45.295349 2791 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:30:45.296200 2791 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:30:45.296275 2791 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:45.296352 2791 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:30:45.296397 2791 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:45.305178 2791 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.195:43109
I20260808 22:30:45.305212 2994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.195:43109 every 8 connection(s)
I20260808 22:30:45.306038 2791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
Aug 08 22:30:45 dist-test-slave-7f01 krb5kdc[2113](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228245, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:30:45.315680 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 2791
I20260808 22:30:45.315863 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance
I20260808 22:30:45.316161 5167 external_mini_cluster.cc:1428] Setting key 935327470b3a459a13a8aa316291f3c0
I20260808 22:30:45.319835 2254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:45.307887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:33941 (local address 127.5.11.254:39537)
0808 22:30:45.308044 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:45.308049 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:45.308833 (+ 784us) server_negotiation.cc:407] Connection header received
0808 22:30:45.310158 (+ 1325us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:45.310161 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:45.310207 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:45.310331 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:45.311587 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:45.312217 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:45.313402 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:45.313618 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:45.313698 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:45.316194 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:45.316198 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:45.316201 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:30:45.316251 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:45.317912 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:45.318476 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:45.318479 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:45.318480 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:45.318559 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:45.318860 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:45.318862 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:45.318863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:45.319037 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:45.319112 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:45.319573 (+ 461us) server_negotiation.cc:299] Negotiation successful
0808 22:30:45.319666 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 22:30:45.320618 3001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:45.308257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:39537 (local address 127.5.11.195:33941)
0808 22:30:45.308684 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:45.308727 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:45.309967 (+ 1240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:45.310525 (+ 558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:45.310532 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:45.310900 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:30:45.311362 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:45.311424 (+ 62us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:45.312338 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:45.312341 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:45.313285 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:45.313295 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:45.313750 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:45.313753 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:45.313899 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:45.314406 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:30:45.314423 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:30:45.316035 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:30:45.318034 (+ 1999us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:30:45.318039 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:30:45.318049 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:30:45.318393 (+ 344us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:30:45.318641 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:30:45.318644 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:30:45.318645 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:30:45.318777 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:30:45.319125 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:30:45.319129 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:30:45.319364 (+ 235us) client_negotiation.cc:769] Sending connection context
0808 22:30:45.319522 (+ 158us) client_negotiation.cc:240] Negotiation successful
0808 22:30:45.319829 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":109,"threads_started":1}
I20260808 22:30:45.321800 2998 heartbeater.cc:347] Connected to a master server at 127.5.11.254:39537
I20260808 22:30:45.322011 2998 heartbeater.cc:464] Registering TS with master...
I20260808 22:30:45.322449 2998 heartbeater.cc:511] Master 127.5.11.254:39537 requested a full tablet report, sending...
I20260808 22:30:45.323407 2187 ts_manager.cc:194] Registered new tserver with Master: eec9f8c31a3b4a30aa57be9bc56348c1 (127.5.11.195:43109)
I20260808 22:30:45.324101 2187 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.195@KRBTEST.COM'} at 127.5.11.195:33941
I20260808 22:30:45.330063 5167 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:30:45.339495 2254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:45.332166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34866 (local address 127.5.11.254:39537)
0808 22:30:45.332283 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:45.332286 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:45.332509 (+ 223us) server_negotiation.cc:407] Connection header received
0808 22:30:45.332605 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:45.332611 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:45.332681 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:45.332764 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:45.333418 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:45.333896 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:45.334691 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:45.334889 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:45.334960 (+ 71us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:45.336556 (+ 1596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:45.336559 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:45.336561 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:30:45.336583 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:45.338025 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:45.338468 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:45.338471 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:45.338473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:45.338545 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:45.338810 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:45.338813 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:45.338814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:45.339005 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:45.339117 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:45.339310 (+ 193us) server_negotiation.cc:299] Negotiation successful
0808 22:30:45.339383 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 22:30:45.342486 2187 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34866:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:30:45.345693 2187 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:30:45.360040 3015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:45.353626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:43109 (local address 127.0.0.1:53134)
0808 22:30:45.354364 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:45.354423 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:45.354549 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:45.355322 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:45.355330 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:45.355355 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:30:45.355500 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:45.355508 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:45.356761 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:45.356766 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:45.358315 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:45.358481 (+ 166us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:45.359442 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:45.359446 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:45.359563 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:45.359601 (+ 38us) client_negotiation.cc:769] Sending connection context
0808 22:30:45.359707 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 22:30:45.359845 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":554,"mutex_wait_us":66,"thread_start_us":89,"threads_started":1}
I20260808 22:30:45.360073 3021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:45.354345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:43077 (local address 127.0.0.1:54048)
0808 22:30:45.354770 (+ 425us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:45.354783 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:45.354865 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:45.355724 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:45.355727 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:45.355777 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:30:45.355882 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:45.355889 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:45.357393 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:45.357398 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:45.358332 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:45.358347 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:45.359704 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:45.359708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:45.359821 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:45.359841 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 22:30:45.359904 (+ 63us) client_negotiation.cc:240] Negotiation successful
0808 22:30:45.359960 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"mutex_wait_us":56,"thread_start_us":124,"threads_started":1}
I20260808 22:30:45.360448 3022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:45.354801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54048 (local address 127.5.11.193:43077)
0808 22:30:45.355277 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:45.355291 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:45.355312 (+ 21us) server_negotiation.cc:407] Connection header received
0808 22:30:45.355382 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:45.355387 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:45.355560 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:45.356042 (+ 482us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:30:45.356203 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:45.357254 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:45.358473 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:45.359555 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:45.359700 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:45.359903 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:45.359961 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 22:30:45.360065 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":75,"threads_started":1}
I20260808 22:30:45.360708 3016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:45.353823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53134 (local address 127.5.11.195:43109)
0808 22:30:45.354666 (+ 843us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:45.354678 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:45.354701 (+ 23us) server_negotiation.cc:407] Connection header received
0808 22:30:45.354779 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:45.354784 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:45.355004 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:45.355187 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:30:45.355790 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:45.356634 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:45.358561 (+ 1927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:45.359321 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:45.359445 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:45.359593 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:45.360430 (+ 837us) server_negotiation.cc:299] Negotiation successful
0808 22:30:45.360546 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":729,"thread_start_us":171,"threads_started":1}
I20260808 22:30:45.360708 3020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:45.354186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55086 (local address 127.5.11.194:36513)
0808 22:30:45.355059 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:45.355073 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:45.355098 (+ 25us) server_negotiation.cc:407] Connection header received
0808 22:30:45.355177 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:45.355182 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:45.355389 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:45.355539 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:30:45.355723 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:45.356867 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:45.358519 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:45.359511 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:45.359647 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:45.360396 (+ 749us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:45.360454 (+ 58us) server_negotiation.cc:299] Negotiation successful
0808 22:30:45.360555 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":763,"thread_start_us":139,"threads_started":1}
I20260808 22:30:45.360983 3017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:45.354008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:36513 (local address 127.0.0.1:55086)
0808 22:30:45.354366 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:45.354424 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:45.354550 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:45.355529 (+ 979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:45.355535 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:45.355558 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:30:45.355648 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:45.355654 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:45.357007 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:45.357010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:45.358383 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:45.358393 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:45.359830 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:45.359834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:45.359937 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:45.359961 (+ 24us) client_negotiation.cc:769] Sending connection context
0808 22:30:45.360027 (+ 66us) client_negotiation.cc:240] Negotiation successful
0808 22:30:45.360090 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"mutex_wait_us":205,"spinlock_wait_cycles":896,"thread_start_us":106,"threads_started":1}
I20260808 22:30:45.363543 2887 tablet_service.cc:1467] Processing CreateTablet for tablet 6c43328ddf04482989c4d1674fca2f23 (DEFAULT_TABLE table=test-table [id=21c545fbc28541c2a2c03bccda0e5e37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:30:45.363569 2659 tablet_service.cc:1467] Processing CreateTablet for tablet 6c43328ddf04482989c4d1674fca2f23 (DEFAULT_TABLE table=test-table [id=21c545fbc28541c2a2c03bccda0e5e37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:30:45.363564 2405 tablet_service.cc:1467] Processing CreateTablet for tablet 6c43328ddf04482989c4d1674fca2f23 (DEFAULT_TABLE table=test-table [id=21c545fbc28541c2a2c03bccda0e5e37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:30:45.364635 2887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c43328ddf04482989c4d1674fca2f23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:30:45.364640 2659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c43328ddf04482989c4d1674fca2f23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:30:45.364640 2405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c43328ddf04482989c4d1674fca2f23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:30:45.371274 3023 tablet_bootstrap.cc:492] T 6c43328ddf04482989c4d1674fca2f23 P eec9f8c31a3b4a30aa57be9bc56348c1: Bootstrap starting.
I20260808 22:30:45.372571 3024 tablet_bootstrap.cc:492] T 6c43328ddf04482989c4d1674fca2f23 P ce434990992f431e8caa51c180ee2693: Bootstrap starting.
I20260808 22:30:45.374254 3023 tablet_bootstrap.cc:654] T 6c43328ddf04482989c4d1674fca2f23 P eec9f8c31a3b4a30aa57be9bc56348c1: Neither blocks nor log segments found. Creating new log.
I20260808 22:30:45.374850 3024 tablet_bootstrap.cc:654] T 6c43328ddf04482989c4d1674fca2f23 P ce434990992f431e8caa51c180ee2693: Neither blocks nor log segments found. Creating new log.
I20260808 22:30:45.375717 3024 log.cc:824] T 6c43328ddf04482989c4d1674fca2f23 P ce434990992f431e8caa51c180ee2693: Log is configured to *not* fsync() on all Append() calls
I20260808 22:30:45.375723 3023 log.cc:824] T 6c43328ddf04482989c4d1674fca2f23 P eec9f8c31a3b4a30aa57be9bc56348c1: Log is configured to *not* fsync() on all Append() calls
I20260808 22:30:45.376129 3025 tablet_bootstrap.cc:492] T 6c43328ddf04482989c4d1674fca2f23 P d62e7dd81270464d92448d7a1ab802da: Bootstrap starting.
I20260808 22:30:45.377857 3023 tablet_bootstrap.cc:492] T 6c43328ddf04482989c4d1674fca2f23 P eec9f8c31a3b4a30aa57be9bc56348c1: No bootstrap required, opened a new log
I20260808 22:30:45.378068 3023 ts_tablet_manager.cc:1397] T 6c43328ddf04482989c4d1674fca2f23 P eec9f8c31a3b4a30aa57be9bc56348c1: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260808 22:30:45.378333 3025 tablet_bootstrap.cc:654] T 6c43328ddf04482989c4d1674fca2f23 P d62e7dd81270464d92448d7a1ab802da: Neither blocks nor log segments found. Creating new log.
I20260808 22:30:45.378360 3024 tablet_bootstrap.cc:492] T 6c43328ddf04482989c4d1674fca2f23 P ce434990992f431e8caa51c180ee2693: No bootstrap required, opened a new log
I20260808 22:30:45.378542 3024 ts_tablet_manager.cc:1397] T 6c43328ddf04482989c4d1674fca2f23 P ce434990992f431e8caa51c180ee2693: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260808 22:30:45.379664 3025 log.cc:824] T 6c43328ddf04482989c4d1674fca2f23 P d62e7dd81270464d92448d7a1ab802da: Log is configured to *not* fsync() on all Append() calls
I20260808 22:30:45.381438 3023 raft_consensus.cc:348] T 6c43328ddf04482989c4d1674fca2f23 P eec9f8c31a3b4a30aa57be9bc56348c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eec9f8c31a3b4a30aa57be9bc56348c1" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 43109 } } peers { permanent_uuid: "ce434990992f431e8caa51c180ee2693" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43077 } } peers { permanent_uuid: "d62e7dd81270464d92448d7a1ab802da" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36513 } }
I20260808 22:30:45.381695 3023 raft_consensus.cc:374] T 6c43328ddf04482989c4d1674fca2f23 P eec9f8c31a3b4a30aa57be9bc56348c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:30:45.381758 3023 raft_consensus.cc:729] T 6c43328ddf04482989c4d1674fca2f23 P eec9f8c31a3b4a30aa57be9bc56348c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eec9f8c31a3b4a30aa57be9bc56348c1, State: Initialized, Role: FOLLOWER
I20260808 22:30:45.381906 3025 tablet_bootstrap.cc:492] T 6c43328ddf04482989c4d1674fca2f23 P d62e7dd81270464d92448d7a1ab802da: No bootstrap required, opened a new log
I20260808 22:30:45.382097 3025 ts_tablet_manager.cc:1397] T 6c43328ddf04482989c4d1674fca2f23 P d62e7dd81270464d92448d7a1ab802da: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260808 22:30:45.382288 3023 consensus_queue.cc:260] T 6c43328ddf04482989c4d1674fca2f23 P eec9f8c31a3b4a30aa57be9bc56348c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eec9f8c31a3b4a30aa57be9bc56348c1" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 43109 } } peers { permanent_uuid: "ce434990992f431e8caa51c180ee2693" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43077 } } peers { permanent_uuid: "d62e7dd81270464d92448d7a1ab802da" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36513 } }
I20260808 22:30:45.382951 2998 heartbeater.cc:503] Master 127.5.11.254:39537 was elected leader, sending a full tablet report...
I20260808 22:30:45.383046 3024 raft_consensus.cc:348] T 6c43328ddf04482989c4d1674fca2f23 P ce434990992f431e8caa51c180ee2693 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eec9f8c31a3b4a30aa57be9bc56348c1" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 43109 } } peers { permanent_uuid: "ce434990992f431e8caa51c180ee2693" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43077 } } peers { permanent_uuid: "d62e7dd81270464d92448d7a1ab802da" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36513 } }
I20260808 22:30:45.383535 3024 raft_consensus.cc:374] T 6c43328ddf04482989c4d1674fca2f23 P ce434990992f431e8caa51c180ee2693 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:30:45.383592 3024 raft_consensus.cc:729] T 6c43328ddf04482989c4d1674fca2f23 P ce434990992f431e8caa51c180ee2693 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce434990992f431e8caa51c180ee2693, State: Initialized, Role: FOLLOWER
I20260808 22:30:45.384368 3024 consensus_queue.cc:260] T 6c43328ddf04482989c4d1674fca2f23 P ce434990992f431e8caa51c180ee2693 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eec9f8c31a3b4a30aa57be9bc56348c1" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 43109 } } peers { permanent_uuid: "ce434990992f431e8caa51c180ee2693" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43077 } } peers { permanent_uuid: "d62e7dd81270464d92448d7a1ab802da" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36513 } }
I20260808 22:30:45.384579 3023 ts_tablet_manager.cc:1428] T 6c43328ddf04482989c4d1674fca2f23 P eec9f8c31a3b4a30aa57be9bc56348c1: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 22:30:45.385250 2532 heartbeater.cc:503] Master 127.5.11.254:39537 was elected leader, sending a full tablet report...
I20260808 22:30:45.385345 3024 ts_tablet_manager.cc:1428] T 6c43328ddf04482989c4d1674fca2f23 P ce434990992f431e8caa51c180ee2693: Time spent starting tablet: real 0.007s user 0.005s sys 0.002s
I20260808 22:30:45.385878 3025 raft_consensus.cc:348] T 6c43328ddf04482989c4d1674fca2f23 P d62e7dd81270464d92448d7a1ab802da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eec9f8c31a3b4a30aa57be9bc56348c1" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 43109 } } peers { permanent_uuid: "ce434990992f431e8caa51c180ee2693" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43077 } } peers { permanent_uuid: "d62e7dd81270464d92448d7a1ab802da" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36513 } }
I20260808 22:30:45.386140 3025 raft_consensus.cc:374] T 6c43328ddf04482989c4d1674fca2f23 P d62e7dd81270464d92448d7a1ab802da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:30:45.386200 3025 raft_consensus.cc:729] T 6c43328ddf04482989c4d1674fca2f23 P d62e7dd81270464d92448d7a1ab802da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d62e7dd81270464d92448d7a1ab802da, State: Initialized, Role: FOLLOWER
I20260808 22:30:45.386605 3025 consensus_queue.cc:260] T 6c43328ddf04482989c4d1674fca2f23 P d62e7dd81270464d92448d7a1ab802da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eec9f8c31a3b4a30aa57be9bc56348c1" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 43109 } } peers { permanent_uuid: "ce434990992f431e8caa51c180ee2693" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43077 } } peers { permanent_uuid: "d62e7dd81270464d92448d7a1ab802da" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36513 } }
I20260808 22:30:45.387339 2777 heartbeater.cc:503] Master 127.5.11.254:39537 was elected leader, sending a full tablet report...
I20260808 22:30:45.387573 3025 ts_tablet_manager.cc:1428] T 6c43328ddf04482989c4d1674fca2f23 P d62e7dd81270464d92448d7a1ab802da: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260808 22:30:45.387874 3031 raft_consensus.cc:482] T 6c43328ddf04482989c4d1674fca2f23 P ce434990992f431e8caa51c180ee2693 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:30:45.388026 3031 raft_consensus.cc:504] T 6c43328ddf04482989c4d1674fca2f23 P ce434990992f431e8caa51c180ee2693 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eec9f8c31a3b4a30aa57be9bc56348c1" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 43109 } } peers { permanent_uuid: "ce434990992f431e8caa51c180ee2693" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43077 } } peers { permanent_uuid: "d62e7dd81270464d92448d7a1ab802da" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36513 } }
I20260808 22:30:45.389302 3031 leader_election.cc:288] T 6c43328ddf04482989c4d1674fca2f23 P ce434990992f431e8caa51c180ee2693 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eec9f8c31a3b4a30aa57be9bc56348c1 (127.5.11.195:43109), d62e7dd81270464d92448d7a1ab802da (127.5.11.194:36513)
I20260808 22:30:45.393755 3016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:45.389520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:56865 (local address 127.5.11.195:43109)
0808 22:30:45.389644 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:45.389649 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:45.389916 (+ 267us) server_negotiation.cc:407] Connection header received
0808 22:30:45.390066 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:45.390069 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:45.390118 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:45.390196 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:30:45.390531 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:45.391165 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:45.392682 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:45.393111 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:45.393249 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:45.393302 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:45.393558 (+ 256us) server_negotiation.cc:299] Negotiation successful
0808 22:30:45.393624 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 22:30:45.393766 3020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:45.390020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:37275 (local address 127.5.11.194:36513)
0808 22:30:45.390153 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:45.390158 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:45.390172 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:30:45.390216 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:45.390219 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:45.390256 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:45.390325 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:30:45.390571 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:45.391160 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:45.392696 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:45.393117 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:45.393253 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:45.393307 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:45.393580 (+ 273us) server_negotiation.cc:299] Negotiation successful
0808 22:30:45.393649 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 22:30:45.393810 3040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:45.389474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:43109 (local address 127.5.11.193:56865)
0808 22:30:45.389809 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:45.389822 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:45.389961 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:45.390227 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:45.390229 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:45.390244 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:30:45.390403 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:45.390410 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:45.391310 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:45.391314 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:45.392554 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:45.392564 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:45.393249 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:45.393252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:45.393411 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:45.393448 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 22:30:45.393541 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 22:30:45.393624 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":53,"thread_start_us":87,"threads_started":1}
I20260808 22:30:45.393817 3041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:45.389643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:36513 (local address 127.5.11.193:37275)
0808 22:30:45.389923 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:45.389937 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:45.390048 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:45.390347 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:45.390352 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:45.390368 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:30:45.390472 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:45.390482 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:45.391270 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:45.391273 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:45.392571 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:45.392581 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:45.393253 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:45.393257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:45.393445 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:45.393464 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 22:30:45.393562 (+ 98us) client_negotiation.cc:240] Negotiation successful
0808 22:30:45.393644 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":130,"thread_start_us":127,"threads_started":1}
I20260808 22:30:45.394446 2919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c43328ddf04482989c4d1674fca2f23" candidate_uuid: "ce434990992f431e8caa51c180ee2693" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eec9f8c31a3b4a30aa57be9bc56348c1" is_pre_election: true
I20260808 22:30:45.394483 2694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c43328ddf04482989c4d1674fca2f23" candidate_uuid: "ce434990992f431e8caa51c180ee2693" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62e7dd81270464d92448d7a1ab802da" is_pre_election: true
I20260808 22:30:45.394721 2919 raft_consensus.cc:2454] T 6c43328ddf04482989c4d1674fca2f23 P eec9f8c31a3b4a30aa57be9bc56348c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce434990992f431e8caa51c180ee2693 in term 0.
I20260808 22:30:45.394734 2694 raft_consensus.cc:2454] T 6c43328ddf04482989c4d1674fca2f23 P d62e7dd81270464d92448d7a1ab802da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ce434990992f431e8caa51c180ee2693 in term 0.
I20260808 22:30:45.395226 2334 leader_election.cc:302] T 6c43328ddf04482989c4d1674fca2f23 P ce434990992f431e8caa51c180ee2693 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce434990992f431e8caa51c180ee2693, d62e7dd81270464d92448d7a1ab802da; no voters:
I20260808 22:30:45.395577 3032 raft_consensus.cc:2781] T 6c43328ddf04482989c4d1674fca2f23 P ce434990992f431e8caa51c180ee2693 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:30:45.395699 3032 raft_consensus.cc:482] T 6c43328ddf04482989c4d1674fca2f23 P ce434990992f431e8caa51c180ee2693 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:30:45.395803 3032 raft_consensus.cc:3037] T 6c43328ddf04482989c4d1674fca2f23 P ce434990992f431e8caa51c180ee2693 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:30:45.396903 3032 raft_consensus.cc:504] T 6c43328ddf04482989c4d1674fca2f23 P ce434990992f431e8caa51c180ee2693 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eec9f8c31a3b4a30aa57be9bc56348c1" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 43109 } } peers { permanent_uuid: "ce434990992f431e8caa51c180ee2693" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43077 } } peers { permanent_uuid: "d62e7dd81270464d92448d7a1ab802da" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36513 } }
I20260808 22:30:45.397189 3032 leader_election.cc:288] T 6c43328ddf04482989c4d1674fca2f23 P ce434990992f431e8caa51c180ee2693 [CANDIDATE]: Term 1 election: Requested vote from peers eec9f8c31a3b4a30aa57be9bc56348c1 (127.5.11.195:43109), d62e7dd81270464d92448d7a1ab802da (127.5.11.194:36513)
I20260808 22:30:45.397594 2694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c43328ddf04482989c4d1674fca2f23" candidate_uuid: "ce434990992f431e8caa51c180ee2693" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d62e7dd81270464d92448d7a1ab802da"
I20260808 22:30:45.397594 2919 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c43328ddf04482989c4d1674fca2f23" candidate_uuid: "ce434990992f431e8caa51c180ee2693" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eec9f8c31a3b4a30aa57be9bc56348c1"
I20260808 22:30:45.397725 2694 raft_consensus.cc:3037] T 6c43328ddf04482989c4d1674fca2f23 P d62e7dd81270464d92448d7a1ab802da [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:30:45.397725 2919 raft_consensus.cc:3037] T 6c43328ddf04482989c4d1674fca2f23 P eec9f8c31a3b4a30aa57be9bc56348c1 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:30:45.398983 2694 raft_consensus.cc:2454] T 6c43328ddf04482989c4d1674fca2f23 P d62e7dd81270464d92448d7a1ab802da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce434990992f431e8caa51c180ee2693 in term 1.
I20260808 22:30:45.398983 2919 raft_consensus.cc:2454] T 6c43328ddf04482989c4d1674fca2f23 P eec9f8c31a3b4a30aa57be9bc56348c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ce434990992f431e8caa51c180ee2693 in term 1.
I20260808 22:30:45.399276 2341 leader_election.cc:302] T 6c43328ddf04482989c4d1674fca2f23 P ce434990992f431e8caa51c180ee2693 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ce434990992f431e8caa51c180ee2693, eec9f8c31a3b4a30aa57be9bc56348c1; no voters:
I20260808 22:30:45.399479 3032 raft_consensus.cc:2781] T 6c43328ddf04482989c4d1674fca2f23 P ce434990992f431e8caa51c180ee2693 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:30:45.399605 3032 raft_consensus.cc:686] T 6c43328ddf04482989c4d1674fca2f23 P ce434990992f431e8caa51c180ee2693 [term 1 LEADER]: Becoming Leader. State: Replica: ce434990992f431e8caa51c180ee2693, State: Running, Role: LEADER
I20260808 22:30:45.399950 3032 consensus_queue.cc:237] T 6c43328ddf04482989c4d1674fca2f23 P ce434990992f431e8caa51c180ee2693 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eec9f8c31a3b4a30aa57be9bc56348c1" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 43109 } } peers { permanent_uuid: "ce434990992f431e8caa51c180ee2693" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43077 } } peers { permanent_uuid: "d62e7dd81270464d92448d7a1ab802da" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36513 } }
I20260808 22:30:45.403193 2186 catalog_manager.cc:5495] T 6c43328ddf04482989c4d1674fca2f23 P ce434990992f431e8caa51c180ee2693 reported cstate change: term changed from 0 to 1, leader changed from <none> to ce434990992f431e8caa51c180ee2693 (127.5.11.193). New cstate: current_term: 1 leader_uuid: "ce434990992f431e8caa51c180ee2693" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eec9f8c31a3b4a30aa57be9bc56348c1" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 43109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ce434990992f431e8caa51c180ee2693" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 43077 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d62e7dd81270464d92448d7a1ab802da" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36513 } health_report { overall_health: UNKNOWN } } }
I20260808 22:30:45.424077 3022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:45.420453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54056 (local address 127.5.11.193:43077)
0808 22:30:45.420601 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:45.420606 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:45.420620 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:30:45.420783 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:45.420785 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:45.420842 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:45.420956 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:30:45.421210 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:45.421715 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:45.422593 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:45.422816 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:45.422929 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:45.423236 (+ 307us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:30:45.423621 (+ 385us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:30:45.423801 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:45.423901 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 22:30:45.423936 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 22:30:45.430160 2919 raft_consensus.cc:1261] T 6c43328ddf04482989c4d1674fca2f23 P eec9f8c31a3b4a30aa57be9bc56348c1 [term 1 FOLLOWER]: Refusing update from remote peer ce434990992f431e8caa51c180ee2693: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:30:45.430182 2694 raft_consensus.cc:1261] T 6c43328ddf04482989c4d1674fca2f23 P d62e7dd81270464d92448d7a1ab802da [term 1 FOLLOWER]: Refusing update from remote peer ce434990992f431e8caa51c180ee2693: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:30:45.431006 3032 consensus_queue.cc:1035] T 6c43328ddf04482989c4d1674fca2f23 P ce434990992f431e8caa51c180ee2693 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d62e7dd81270464d92448d7a1ab802da" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 36513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:30:45.431210 3031 consensus_queue.cc:1035] T 6c43328ddf04482989c4d1674fca2f23 P ce434990992f431e8caa51c180ee2693 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eec9f8c31a3b4a30aa57be9bc56348c1" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 43109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:30:45.439169 3044 mvcc.cc:205] Tried to move back new op lower bound from 7316390893273796608 to 7316390893160919040. Current Snapshot: MvccSnapshot[applied={T|T < 7316390893273796608}]
I20260808 22:30:45.440727 3048 mvcc.cc:205] Tried to move back new op lower bound from 7316390893273796608 to 7316390893160919040. Current Snapshot: MvccSnapshot[applied={T|T < 7316390893273796608}]
I20260808 22:30:45.447389 2186 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:30:45.447539 2186 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34866:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:30:45.450063 2186 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2b76ea99d7464072a72107b1836615e8: Sending DeleteTablet for 3 replicas of tablet 6c43328ddf04482989c4d1674fca2f23
I20260808 22:30:45.450961 2659 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c43328ddf04482989c4d1674fca2f23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:30:45 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:55086
I20260808 22:30:45.450959 2405 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c43328ddf04482989c4d1674fca2f23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:30:45 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:54048
I20260808 22:30:45.450980 2887 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c43328ddf04482989c4d1674fca2f23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:30:45 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:53134
I20260808 22:30:45.451490 3061 tablet_replica.cc:330] T 6c43328ddf04482989c4d1674fca2f23 P eec9f8c31a3b4a30aa57be9bc56348c1: stopping tablet replica
I20260808 22:30:45.451802 3057 tablet_replica.cc:330] T 6c43328ddf04482989c4d1674fca2f23 P d62e7dd81270464d92448d7a1ab802da: stopping tablet replica
I20260808 22:30:45.451833 3061 raft_consensus.cc:2229] T 6c43328ddf04482989c4d1674fca2f23 P eec9f8c31a3b4a30aa57be9bc56348c1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:30:45.452092 3057 raft_consensus.cc:2229] T 6c43328ddf04482989c4d1674fca2f23 P d62e7dd81270464d92448d7a1ab802da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:30:45.452139 3061 raft_consensus.cc:2258] T 6c43328ddf04482989c4d1674fca2f23 P eec9f8c31a3b4a30aa57be9bc56348c1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:30:45.452379 3057 raft_consensus.cc:2258] T 6c43328ddf04482989c4d1674fca2f23 P d62e7dd81270464d92448d7a1ab802da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:30:45.452498 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 2298
I20260808 22:30:45.453859 3061 ts_tablet_manager.cc:1905] T 6c43328ddf04482989c4d1674fca2f23 P eec9f8c31a3b4a30aa57be9bc56348c1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:30:45.457315 3061 ts_tablet_manager.cc:1918] T 6c43328ddf04482989c4d1674fca2f23 P eec9f8c31a3b4a30aa57be9bc56348c1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:30:45.457434 3061 log.cc:1196] T 6c43328ddf04482989c4d1674fca2f23 P eec9f8c31a3b4a30aa57be9bc56348c1: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-2/wal/wals/6c43328ddf04482989c4d1674fca2f23
I20260808 22:30:45.457847 3061 ts_tablet_manager.cc:1939] T 6c43328ddf04482989c4d1674fca2f23 P eec9f8c31a3b4a30aa57be9bc56348c1: Deleting consensus metadata
I20260808 22:30:45.460989 2169 catalog_manager.cc:4841] TS eec9f8c31a3b4a30aa57be9bc56348c1 (127.5.11.195:43109): tablet 6c43328ddf04482989c4d1674fca2f23 (table test-table [id=21c545fbc28541c2a2c03bccda0e5e37]) successfully deleted
I20260808 22:30:45.467540 3057 ts_tablet_manager.cc:1905] T 6c43328ddf04482989c4d1674fca2f23 P d62e7dd81270464d92448d7a1ab802da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:30:45.470229 3057 ts_tablet_manager.cc:1918] T 6c43328ddf04482989c4d1674fca2f23 P d62e7dd81270464d92448d7a1ab802da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:30:45.470329 3057 log.cc:1196] T 6c43328ddf04482989c4d1674fca2f23 P d62e7dd81270464d92448d7a1ab802da: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786227991988852-5167-0/minicluster-data/ts-1/wal/wals/6c43328ddf04482989c4d1674fca2f23
I20260808 22:30:45.470690 3057 ts_tablet_manager.cc:1939] T 6c43328ddf04482989c4d1674fca2f23 P d62e7dd81270464d92448d7a1ab802da: Deleting consensus metadata
I20260808 22:30:45.471922 2166 catalog_manager.cc:4841] TS d62e7dd81270464d92448d7a1ab802da (127.5.11.194:36513): tablet 6c43328ddf04482989c4d1674fca2f23 (table test-table [id=21c545fbc28541c2a2c03bccda0e5e37]) successfully deleted
W20260808 22:30:45.473647 2169 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.11.193:43077 (error 108)
I20260808 22:30:45.473937 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 2544
I20260808 22:30:45.480931 3017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:45.480550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:43077 (local address 127.0.0.1:54072)
0808 22:30:45.480728 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:45.480820 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.193:43077: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 22:30:45.481240 2169 catalog_manager.cc:4584] TS ce434990992f431e8caa51c180ee2693 (127.5.11.193:43077): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6c43328ddf04482989c4d1674fca2f23: Network error: Client connection negotiation failed: client connection to 127.5.11.193:43077: connect: Connection refused (error 111)
I20260808 22:30:45.490813 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 2791
I20260808 22:30:45.504942 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 2138
2026-08-08T22:30:45Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2504 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-08T22:30:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:30:45Z Disabled control of system clock
I20260808 22:30:45.543696 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:36889
--webserver_interface=127.5.11.254
--webserver_port=0
--builtin_ntp_servers=127.5.11.212:43811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:36889
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 22:30:45.654989 3070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:30:45.655224 3070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:30:45.655257 3070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:30:45.659119 3070 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:30:45.659168 3070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:30:45.659183 3070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:30:45.659200 3070 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:30:45.659215 3070 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:30:45.663691 3070 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:43811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:36889
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:36889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:30:45.664620 3070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:30:45.665870 3070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:30:45.674130 3078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:45.674188 3079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:45.674249 3070 server_base.cc:1034] running on GCE node
W20260808 22:30:45.674188 3081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:45.674926 3070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:30:45.675961 3070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:30:45.677120 3070 hybrid_clock.cc:648] HybridClock initialized: now 1786228245677103 us; error 30 us; skew 500 ppm
I20260808 22:30:45.678948 3070 webserver.cc:460] Webserver started at http://127.5.11.254:40333/ using document root <none> and password file <none>
I20260808 22:30:45.679461 3070 fs_manager.cc:359] Metadata directory not provided
I20260808 22:30:45.679526 3070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:30:45.679685 3070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:30:45.681813 3070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "063565472c8340d28d039ff28b8a4ee7"
format_stamp: "Formatted at 2026-08-08 22:30:45 on dist-test-slave-7f01"
server_key: "ae2db99b89f369175e611da37705adf0"
server_key_iv: "bebb3b284911d614628804509b577ddd"
server_key_version: "encryptionkey@0"
I20260808 22:30:45.682315 3070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "063565472c8340d28d039ff28b8a4ee7"
format_stamp: "Formatted at 2026-08-08 22:30:45 on dist-test-slave-7f01"
server_key: "ae2db99b89f369175e611da37705adf0"
server_key_iv: "bebb3b284911d614628804509b577ddd"
server_key_version: "encryptionkey@0"
I20260808 22:30:45.685791 3070 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 22:30:45.688058 3091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:45.689188 3070 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 22:30:45.689316 3070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "063565472c8340d28d039ff28b8a4ee7"
format_stamp: "Formatted at 2026-08-08 22:30:45 on dist-test-slave-7f01"
server_key: "ae2db99b89f369175e611da37705adf0"
server_key_iv: "bebb3b284911d614628804509b577ddd"
server_key_version: "encryptionkey@0"
I20260808 22:30:45.689415 3070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:30:45.700577 3070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:30:45.701164 3070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:30:45.701354 3070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:30:45.708097 3070 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:36889
I20260808 22:30:45.708163 3175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:36889 every 8 connection(s)
I20260808 22:30:45.709012 3070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:30:45.709569 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 3070
I20260808 22:30:45.709713 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:30:45.709995 5167 external_mini_cluster.cc:1428] Setting key 840793b1a3d9433d744b37895d2f87da
I20260808 22:30:45.713145 3176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:30:45.718709 3176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 063565472c8340d28d039ff28b8a4ee7: Bootstrap starting.
I20260808 22:30:45.718878 3183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:45.711266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33474 (local address 127.5.11.254:36889)
0808 22:30:45.711923 (+ 657us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:45.711940 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:45.711976 (+ 36us) server_negotiation.cc:407] Connection header received
0808 22:30:45.712600 (+ 624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:45.712618 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:45.713020 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:45.713351 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:45.713788 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:45.714746 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:45.715623 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:45.716157 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:45.716336 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:45.717210 (+ 874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:45.717215 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:45.717227 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:30:45.717589 (+ 362us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:45.717643 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:30:45.717663 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:45.717772 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:45.717944 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 22:30:45.718186 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":115,"threads_started":1}
I20260808 22:30:45.721144 3176 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 063565472c8340d28d039ff28b8a4ee7: Neither blocks nor log segments found. Creating new log.
I20260808 22:30:45.721860 3176 log.cc:824] T 00000000000000000000000000000000 P 063565472c8340d28d039ff28b8a4ee7: Log is configured to *not* fsync() on all Append() calls
I20260808 22:30:45.723888 3176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 063565472c8340d28d039ff28b8a4ee7: No bootstrap required, opened a new log
I20260808 22:30:45.727166 3176 raft_consensus.cc:348] T 00000000000000000000000000000000 P 063565472c8340d28d039ff28b8a4ee7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "063565472c8340d28d039ff28b8a4ee7" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 36889 } }
I20260808 22:30:45.727378 3176 raft_consensus.cc:374] T 00000000000000000000000000000000 P 063565472c8340d28d039ff28b8a4ee7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:30:45.727461 3176 raft_consensus.cc:729] T 00000000000000000000000000000000 P 063565472c8340d28d039ff28b8a4ee7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 063565472c8340d28d039ff28b8a4ee7, State: Initialized, Role: FOLLOWER
I20260808 22:30:45.727921 3176 consensus_queue.cc:260] T 00000000000000000000000000000000 P 063565472c8340d28d039ff28b8a4ee7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "063565472c8340d28d039ff28b8a4ee7" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 36889 } }
I20260808 22:30:45.728082 3176 raft_consensus.cc:388] T 00000000000000000000000000000000 P 063565472c8340d28d039ff28b8a4ee7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:30:45.728173 3176 raft_consensus.cc:482] T 00000000000000000000000000000000 P 063565472c8340d28d039ff28b8a4ee7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:30:45.728307 3176 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 063565472c8340d28d039ff28b8a4ee7 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:30:45.729199 3176 raft_consensus.cc:504] T 00000000000000000000000000000000 P 063565472c8340d28d039ff28b8a4ee7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "063565472c8340d28d039ff28b8a4ee7" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 36889 } }
I20260808 22:30:45.729538 3176 leader_election.cc:302] T 00000000000000000000000000000000 P 063565472c8340d28d039ff28b8a4ee7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 063565472c8340d28d039ff28b8a4ee7; no voters:
I20260808 22:30:45.729856 3176 leader_election.cc:288] T 00000000000000000000000000000000 P 063565472c8340d28d039ff28b8a4ee7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:30:45.729934 3188 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 063565472c8340d28d039ff28b8a4ee7 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:30:45.730185 3188 raft_consensus.cc:686] T 00000000000000000000000000000000 P 063565472c8340d28d039ff28b8a4ee7 [term 1 LEADER]: Becoming Leader. State: Replica: 063565472c8340d28d039ff28b8a4ee7, State: Running, Role: LEADER
I20260808 22:30:45.730456 3188 consensus_queue.cc:237] T 00000000000000000000000000000000 P 063565472c8340d28d039ff28b8a4ee7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "063565472c8340d28d039ff28b8a4ee7" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 36889 } }
I20260808 22:30:45.731009 3176 sys_catalog.cc:564] T 00000000000000000000000000000000 P 063565472c8340d28d039ff28b8a4ee7 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:30:45.732191 3189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 063565472c8340d28d039ff28b8a4ee7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "063565472c8340d28d039ff28b8a4ee7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "063565472c8340d28d039ff28b8a4ee7" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 36889 } } }
I20260808 22:30:45.732335 3189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 063565472c8340d28d039ff28b8a4ee7 [sys.catalog]: This master's current role is: LEADER
I20260808 22:30:45.732620 3191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 063565472c8340d28d039ff28b8a4ee7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 063565472c8340d28d039ff28b8a4ee7. Latest consensus state: current_term: 1 leader_uuid: "063565472c8340d28d039ff28b8a4ee7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "063565472c8340d28d039ff28b8a4ee7" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 36889 } } }
I20260808 22:30:45.732717 3191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 063565472c8340d28d039ff28b8a4ee7 [sys.catalog]: This master's current role is: LEADER
I20260808 22:30:45.732825 3201 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:30:45.735385 3201 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:30:45.740873 3201 catalog_manager.cc:1347] Generated new cluster ID: 56e509b89e144ff0bb00b100a3723ccc
I20260808 22:30:45.740932 3201 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:30:45.760531 3201 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:30:45.761693 3201 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:30:45.775352 3201 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 063565472c8340d28d039ff28b8a4ee7: Generated new TSK 0
I20260808 22:30:45.776151 3201 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:30:45.784462 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.193:0
--local_ip_for_outbound_sockets=127.5.11.193
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:36889
--builtin_ntp_servers=127.5.11.212:43811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 22:30:45.903995 3220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:30:45.904255 3220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:30:45.904320 3220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:30:45.908217 3220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:30:45.908293 3220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:30:45.908385 3220 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.193
I20260808 22:30:45.913010 3220 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:43811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:36889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.193
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:30:45.914021 3220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:30:45.915369 3220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:30:45.924686 3229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:45.924686 3226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:45.924734 3225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:45.924888 3220 server_base.cc:1034] running on GCE node
I20260808 22:30:45.925395 3220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:30:45.925956 3220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:30:45.927145 3220 hybrid_clock.cc:648] HybridClock initialized: now 1786228245927122 us; error 30 us; skew 500 ppm
I20260808 22:30:45.929104 3220 webserver.cc:460] Webserver started at http://127.5.11.193:34173/ using document root <none> and password file <none>
I20260808 22:30:45.929665 3220 fs_manager.cc:359] Metadata directory not provided
I20260808 22:30:45.929730 3220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:30:45.929978 3220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:30:45.932132 3220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-0/data/instance:
uuid: "e0a8dc515f5440c097b5ad17d5cf216d"
format_stamp: "Formatted at 2026-08-08 22:30:45 on dist-test-slave-7f01"
server_key: "fbb738f85fbce4be0467c927eaa21dcc"
server_key_iv: "e4b0dcd4ee92fedb0d8f8ec9d89b3f46"
server_key_version: "encryptionkey@0"
I20260808 22:30:45.932742 3220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance:
uuid: "e0a8dc515f5440c097b5ad17d5cf216d"
format_stamp: "Formatted at 2026-08-08 22:30:45 on dist-test-slave-7f01"
server_key: "fbb738f85fbce4be0467c927eaa21dcc"
server_key_iv: "e4b0dcd4ee92fedb0d8f8ec9d89b3f46"
server_key_version: "encryptionkey@0"
I20260808 22:30:45.936414 3220 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:30:45.938707 3237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:45.939939 3220 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:30:45.940078 3220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-0/wal
uuid: "e0a8dc515f5440c097b5ad17d5cf216d"
format_stamp: "Formatted at 2026-08-08 22:30:45 on dist-test-slave-7f01"
server_key: "fbb738f85fbce4be0467c927eaa21dcc"
server_key_iv: "e4b0dcd4ee92fedb0d8f8ec9d89b3f46"
server_key_version: "encryptionkey@0"
I20260808 22:30:45.940189 3220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:30:45.961978 3220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:30:45.962610 3220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:30:45.962808 3220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:30:45.963385 3220 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:30:45.964381 3220 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:30:45.964454 3220 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:45.964534 3220 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:30:45.964571 3220 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:45.973497 3220 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.193:35545
I20260808 22:30:45.973564 3451 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.193:35545 every 8 connection(s)
I20260808 22:30:45.974395 3220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
I20260808 22:30:45.982157 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 3220
I20260808 22:30:45.982303 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance
I20260808 22:30:45.982687 5167 external_mini_cluster.cc:1428] Setting key d19d12d27596ce942e4de30dc08837e6
I20260808 22:30:45.983636 3183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:45.976243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:55679 (local address 127.5.11.254:36889)
0808 22:30:45.976385 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:45.976389 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:45.977376 (+ 987us) server_negotiation.cc:407] Connection header received
0808 22:30:45.978736 (+ 1360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:45.978740 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:45.978789 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:45.978871 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:45.979779 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:45.980319 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:45.981588 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:45.981815 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:45.981958 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:45.983124 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:45.983128 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:45.983130 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:30:45.983141 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:45.983160 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:30:45.983168 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:45.983246 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:45.983411 (+ 165us) server_negotiation.cc:299] Negotiation successful
0808 22:30:45.983481 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 22:30:45.984418 3455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:45.976617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:36889 (local address 127.5.11.193:55679)
0808 22:30:45.977255 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:45.977294 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:45.978539 (+ 1245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:45.979030 (+ 491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:45.979036 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:45.979115 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:30:45.979605 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:45.979618 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:45.980471 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:45.980474 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:45.981456 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:45.981472 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:45.981973 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:45.981977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:45.982155 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:45.982859 (+ 704us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:30:45.982861 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:30:45.982900 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:30:45.982963 (+ 63us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:30:45.983248 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:30:45.983261 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:30:45.983270 (+ 9us) client_negotiation.cc:769] Sending connection context
0808 22:30:45.983407 (+ 137us) client_negotiation.cc:240] Negotiation successful
0808 22:30:45.983678 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":128,"threads_started":1}
I20260808 22:30:45.984905 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.194:0
--local_ip_for_outbound_sockets=127.5.11.194
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:36889
--builtin_ntp_servers=127.5.11.212:43811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 22:30:45.985641 3453 heartbeater.cc:347] Connected to a master server at 127.5.11.254:36889
I20260808 22:30:45.985888 3453 heartbeater.cc:464] Registering TS with master...
I20260808 22:30:45.986476 3453 heartbeater.cc:511] Master 127.5.11.254:36889 requested a full tablet report, sending...
I20260808 22:30:45.988281 3123 ts_manager.cc:194] Registered new tserver with Master: e0a8dc515f5440c097b5ad17d5cf216d (127.5.11.193:35545)
I20260808 22:30:45.989889 3123 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.11.193:55679
W20260808 22:30:46.098400 3456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:30:46.098649 3456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:30:46.098685 3456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:30:46.102629 3456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:30:46.102679 3456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:30:46.102728 3456 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.194
I20260808 22:30:46.107235 3456 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:43811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:36889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.194
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:30:46.108199 3456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:30:46.109467 3456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:30:46.118713 3472 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:46.118722 3465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:46.118813 3456 server_base.cc:1034] running on GCE node
W20260808 22:30:46.118722 3464 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:46.119302 3456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:30:46.119957 3456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:30:46.121147 3456 hybrid_clock.cc:648] HybridClock initialized: now 1786228246121128 us; error 29 us; skew 500 ppm
I20260808 22:30:46.122967 3456 webserver.cc:460] Webserver started at http://127.5.11.194:39337/ using document root <none> and password file <none>
I20260808 22:30:46.123487 3456 fs_manager.cc:359] Metadata directory not provided
I20260808 22:30:46.123548 3456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:30:46.123708 3456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:30:46.125694 3456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-1/data/instance:
uuid: "88e13b8cfeb844cf94f46b6ac8f75f81"
format_stamp: "Formatted at 2026-08-08 22:30:46 on dist-test-slave-7f01"
server_key: "01f2c85a813fe7ee990ddcbcba68a460"
server_key_iv: "0dc91d3ff90a51c146129e36c5a73e2a"
server_key_version: "encryptionkey@0"
I20260808 22:30:46.126217 3456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance:
uuid: "88e13b8cfeb844cf94f46b6ac8f75f81"
format_stamp: "Formatted at 2026-08-08 22:30:46 on dist-test-slave-7f01"
server_key: "01f2c85a813fe7ee990ddcbcba68a460"
server_key_iv: "0dc91d3ff90a51c146129e36c5a73e2a"
server_key_version: "encryptionkey@0"
I20260808 22:30:46.130064 3456 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260808 22:30:46.132514 3481 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:46.133786 3456 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:30:46.133880 3456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-1/wal
uuid: "88e13b8cfeb844cf94f46b6ac8f75f81"
format_stamp: "Formatted at 2026-08-08 22:30:46 on dist-test-slave-7f01"
server_key: "01f2c85a813fe7ee990ddcbcba68a460"
server_key_iv: "0dc91d3ff90a51c146129e36c5a73e2a"
server_key_version: "encryptionkey@0"
I20260808 22:30:46.133961 3456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:30:46.146147 3456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:30:46.146662 3456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:30:46.146817 3456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:30:46.147282 3456 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:30:46.148200 3456 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:30:46.148247 3456 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:46.148283 3456 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:30:46.148299 3456 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:46.156859 3456 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.194:33667
I20260808 22:30:46.156889 3675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.194:33667 every 8 connection(s)
I20260808 22:30:46.157663 3456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
I20260808 22:30:46.162715 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 3456
I20260808 22:30:46.162847 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance
I20260808 22:30:46.163165 5167 external_mini_cluster.cc:1428] Setting key 2bd8e270ab15cdc4b327f69690428e4a
I20260808 22:30:46.165257 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.195:0
--local_ip_for_outbound_sockets=127.5.11.195
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:36889
--builtin_ntp_servers=127.5.11.212:43811
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 22:30:46.167181 3183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:46.159391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:60587 (local address 127.5.11.254:36889)
0808 22:30:46.159533 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:46.159537 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:46.160593 (+ 1056us) server_negotiation.cc:407] Connection header received
0808 22:30:46.161888 (+ 1295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:46.161891 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:46.161940 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:46.162015 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:46.162893 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:46.163632 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:46.165084 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:46.165447 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:46.165570 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:46.166591 (+ 1021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:46.166598 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:46.166604 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:30:46.166622 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:46.166648 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:30:46.166662 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:46.166742 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:46.166893 (+ 151us) server_negotiation.cc:299] Negotiation successful
0808 22:30:46.166965 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 22:30:46.168233 3679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:46.159834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:36889 (local address 127.5.11.194:60587)
0808 22:30:46.160453 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:46.160492 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:46.161704 (+ 1212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:46.162169 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:46.162175 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:46.162254 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:30:46.162711 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:46.162722 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:46.163798 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:46.163801 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:46.164956 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:46.164966 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:46.165597 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:46.165600 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:46.165738 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:46.166337 (+ 599us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:30:46.166338 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:30:46.166376 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:30:46.166435 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:30:46.166749 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:30:46.166756 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:30:46.166760 (+ 4us) client_negotiation.cc:769] Sending connection context
0808 22:30:46.166861 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 22:30:46.167073 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":113,"threads_started":1}
I20260808 22:30:46.169370 3676 heartbeater.cc:347] Connected to a master server at 127.5.11.254:36889
I20260808 22:30:46.169615 3676 heartbeater.cc:464] Registering TS with master...
I20260808 22:30:46.170121 3676 heartbeater.cc:511] Master 127.5.11.254:36889 requested a full tablet report, sending...
I20260808 22:30:46.171223 3123 ts_manager.cc:194] Registered new tserver with Master: 88e13b8cfeb844cf94f46b6ac8f75f81 (127.5.11.194:33667)
I20260808 22:30:46.172119 3123 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.11.194:60587
W20260808 22:30:46.278528 3681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:30:46.278759 3681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:30:46.278795 3681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:30:46.282735 3681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:30:46.282783 3681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:30:46.282835 3681 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.195
I20260808 22:30:46.287321 3681 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:43811
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:36889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.195
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:30:46.288252 3681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:30:46.289511 3681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:30:46.298877 3690 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:46.298879 3695 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:46.298964 3692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:46.299266 3681 server_base.cc:1034] running on GCE node
I20260808 22:30:46.299715 3681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:30:46.300369 3681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:30:46.301554 3681 hybrid_clock.cc:648] HybridClock initialized: now 1786228246301544 us; error 30 us; skew 500 ppm
I20260808 22:30:46.303416 3681 webserver.cc:460] Webserver started at http://127.5.11.195:39431/ using document root <none> and password file <none>
I20260808 22:30:46.304071 3681 fs_manager.cc:359] Metadata directory not provided
I20260808 22:30:46.304167 3681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:30:46.304410 3681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:30:46.306411 3681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-2/data/instance:
uuid: "401f2da7203643aeaceb4e1f36f1597f"
format_stamp: "Formatted at 2026-08-08 22:30:46 on dist-test-slave-7f01"
server_key: "53f2e74f4474c577bd35b66ef8e2115d"
server_key_iv: "9b84bf503a7232b3bd058e0524fa0d0e"
server_key_version: "encryptionkey@0"
I20260808 22:30:46.306988 3681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance:
uuid: "401f2da7203643aeaceb4e1f36f1597f"
format_stamp: "Formatted at 2026-08-08 22:30:46 on dist-test-slave-7f01"
server_key: "53f2e74f4474c577bd35b66ef8e2115d"
server_key_iv: "9b84bf503a7232b3bd058e0524fa0d0e"
server_key_version: "encryptionkey@0"
I20260808 22:30:46.310629 3681 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.004s
I20260808 22:30:46.313032 3707 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:46.314186 3681 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:30:46.314334 3681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-2/wal
uuid: "401f2da7203643aeaceb4e1f36f1597f"
format_stamp: "Formatted at 2026-08-08 22:30:46 on dist-test-slave-7f01"
server_key: "53f2e74f4474c577bd35b66ef8e2115d"
server_key_iv: "9b84bf503a7232b3bd058e0524fa0d0e"
server_key_version: "encryptionkey@0"
I20260808 22:30:46.314446 3681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:30:46.335351 3681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:30:46.336033 3681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:30:46.336220 3681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:30:46.336850 3681 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:30:46.337810 3681 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:30:46.337894 3681 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:46.337963 3681 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:30:46.338019 3681 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:46.346870 3681 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.195:36315
I20260808 22:30:46.346925 3901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.195:36315 every 8 connection(s)
I20260808 22:30:46.347724 3681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
I20260808 22:30:46.353252 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 3681
I20260808 22:30:46.353384 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance
I20260808 22:30:46.353653 5167 external_mini_cluster.cc:1428] Setting key 79d8cd656e5eef5d971f9c44d2c83b77
I20260808 22:30:46.357108 3183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:46.349489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:45177 (local address 127.5.11.254:36889)
0808 22:30:46.349634 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:46.349638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:46.350660 (+ 1022us) server_negotiation.cc:407] Connection header received
0808 22:30:46.352082 (+ 1422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:46.352085 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:46.352132 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:46.352207 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:46.353043 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:46.353657 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:46.355068 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:46.355294 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:46.355403 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:46.356609 (+ 1206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:46.356613 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:46.356616 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:30:46.356628 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:46.356655 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:30:46.356663 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:46.356739 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:46.356910 (+ 171us) server_negotiation.cc:299] Negotiation successful
0808 22:30:46.356970 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260808 22:30:46.357889 3907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:46.349902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:36889 (local address 127.5.11.195:45177)
0808 22:30:46.350523 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:46.350562 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:46.351874 (+ 1312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:46.352369 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:46.352375 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:46.352456 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:30:46.352902 (+ 446us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:46.352914 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:46.353785 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:46.353791 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:46.354934 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:46.354949 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:46.355442 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:46.355446 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:46.355655 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:46.356351 (+ 696us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:30:46.356352 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:30:46.356391 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:30:46.356450 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:30:46.356743 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:30:46.356754 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:30:46.356761 (+ 7us) client_negotiation.cc:769] Sending connection context
0808 22:30:46.356895 (+ 134us) client_negotiation.cc:240] Negotiation successful
0808 22:30:46.357156 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":117,"threads_started":1}
I20260808 22:30:46.358917 3904 heartbeater.cc:347] Connected to a master server at 127.5.11.254:36889
I20260808 22:30:46.359155 3904 heartbeater.cc:464] Registering TS with master...
I20260808 22:30:46.359650 3904 heartbeater.cc:511] Master 127.5.11.254:36889 requested a full tablet report, sending...
I20260808 22:30:46.360654 3123 ts_manager.cc:194] Registered new tserver with Master: 401f2da7203643aeaceb4e1f36f1597f (127.5.11.195:36315)
I20260808 22:30:46.361231 3123 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.11.195:45177
I20260808 22:30:46.367547 5167 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:30:46.372900 3183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:46.369509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33488 (local address 127.5.11.254:36889)
0808 22:30:46.369654 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:46.369659 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:46.369810 (+ 151us) server_negotiation.cc:407] Connection header received
0808 22:30:46.369868 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:46.369870 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:46.369907 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:46.369969 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:46.370218 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:46.370686 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:46.371440 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:46.371637 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:46.371709 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:46.372431 (+ 722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:46.372435 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:46.372437 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:30:46.372448 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:46.372465 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:30:46.372472 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:46.372566 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:46.372666 (+ 100us) server_negotiation.cc:299] Negotiation successful
0808 22:30:46.372693 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 22:30:46.375700 3123 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33488:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:30:46.378682 3123 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:30:46.392843 3927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:46.386923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:36315 (local address 127.0.0.1:41220)
0808 22:30:46.387870 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:46.387920 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:46.388054 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:46.388844 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:46.388849 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:46.388866 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:30:46.388974 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:46.388979 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:46.389966 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:46.389970 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:46.391169 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:46.391180 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:46.392272 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:46.392276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:46.392391 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:46.392436 (+ 45us) client_negotiation.cc:769] Sending connection context
0808 22:30:46.392584 (+ 148us) client_negotiation.cc:240] Negotiation successful
0808 22:30:46.392705 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":737,"thread_start_us":87,"threads_started":1}
I20260808 22:30:46.393002 3932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:46.387083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:33667 (local address 127.0.0.1:49194)
0808 22:30:46.388010 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:46.388024 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:46.388123 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:46.388747 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:46.388752 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:46.388777 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:30:46.388911 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:46.388918 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:46.390468 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:46.390471 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:46.391628 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:46.391638 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:46.392595 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:46.392599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:46.392686 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:46.392723 (+ 37us) client_negotiation.cc:769] Sending connection context
0808 22:30:46.392788 (+ 65us) client_negotiation.cc:240] Negotiation successful
0808 22:30:46.392856 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":867,"mutex_wait_us":26,"thread_start_us":97,"threads_started":1}
I20260808 22:30:46.393146 3935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:46.387731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49194 (local address 127.5.11.194:33667)
0808 22:30:46.388211 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:46.388224 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:46.388248 (+ 24us) server_negotiation.cc:407] Connection header received
0808 22:30:46.388330 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:46.388338 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:46.388560 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:46.389077 (+ 517us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:30:46.389217 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:46.390339 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:46.391782 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:46.392408 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:46.392610 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:46.392779 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:46.392832 (+ 53us) server_negotiation.cc:299] Negotiation successful
0808 22:30:46.392934 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":113,"threads_started":1}
I20260808 22:30:46.393395 3929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:46.387105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41220 (local address 127.5.11.195:36315)
0808 22:30:46.387454 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:46.387468 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:46.388212 (+ 744us) server_negotiation.cc:407] Connection header received
0808 22:30:46.388303 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:46.388309 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:46.388528 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:46.388686 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:30:46.389093 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:46.389844 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:46.391313 (+ 1469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:46.392141 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:46.392246 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:46.392435 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:46.393120 (+ 685us) server_negotiation.cc:299] Negotiation successful
0808 22:30:46.393232 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":171,"threads_started":1}
I20260808 22:30:46.393153 3928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:46.387669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:35545 (local address 127.0.0.1:55170)
0808 22:30:46.388306 (+ 637us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:46.388319 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:46.388394 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:46.389107 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:46.389110 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:46.389127 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:30:46.389203 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:46.389208 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:46.390643 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:46.390647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:46.391744 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:46.391777 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:46.392812 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:46.392815 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:46.392902 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:46.392922 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 22:30:46.392971 (+ 49us) client_negotiation.cc:240] Negotiation successful
0808 22:30:46.393034 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"mutex_wait_us":19,"thread_start_us":115,"threads_started":1}
I20260808 22:30:46.393764 3936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:46.388154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55170 (local address 127.5.11.193:35545)
0808 22:30:46.388660 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:46.388674 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:46.388698 (+ 24us) server_negotiation.cc:407] Connection header received
0808 22:30:46.388771 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:46.388776 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:46.388959 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:46.389346 (+ 387us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:30:46.389487 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:46.390533 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:46.391898 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:46.392670 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:46.393253 (+ 583us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:46.393437 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:46.393513 (+ 76us) server_negotiation.cc:299] Negotiation successful
0808 22:30:46.393606 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":109,"threads_started":1}
I20260808 22:30:46.396351 3567 tablet_service.cc:1467] Processing CreateTablet for tablet 2155e8935c0d4607a473779d9e6e2aa9 (DEFAULT_TABLE table=test-table [id=b7a3d0ea51604c52ab9338180968ab59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:30:46.396499 3794 tablet_service.cc:1467] Processing CreateTablet for tablet 2155e8935c0d4607a473779d9e6e2aa9 (DEFAULT_TABLE table=test-table [id=b7a3d0ea51604c52ab9338180968ab59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:30:46.396499 3319 tablet_service.cc:1467] Processing CreateTablet for tablet 2155e8935c0d4607a473779d9e6e2aa9 (DEFAULT_TABLE table=test-table [id=b7a3d0ea51604c52ab9338180968ab59]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:30:46.397362 3567 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2155e8935c0d4607a473779d9e6e2aa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:30:46.397521 3794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2155e8935c0d4607a473779d9e6e2aa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:30:46.397521 3319 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2155e8935c0d4607a473779d9e6e2aa9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:30:46.403822 3937 tablet_bootstrap.cc:492] T 2155e8935c0d4607a473779d9e6e2aa9 P 88e13b8cfeb844cf94f46b6ac8f75f81: Bootstrap starting.
I20260808 22:30:46.405081 3940 tablet_bootstrap.cc:492] T 2155e8935c0d4607a473779d9e6e2aa9 P e0a8dc515f5440c097b5ad17d5cf216d: Bootstrap starting.
I20260808 22:30:46.406054 3937 tablet_bootstrap.cc:654] T 2155e8935c0d4607a473779d9e6e2aa9 P 88e13b8cfeb844cf94f46b6ac8f75f81: Neither blocks nor log segments found. Creating new log.
I20260808 22:30:46.406791 3937 log.cc:824] T 2155e8935c0d4607a473779d9e6e2aa9 P 88e13b8cfeb844cf94f46b6ac8f75f81: Log is configured to *not* fsync() on all Append() calls
I20260808 22:30:46.406852 3940 tablet_bootstrap.cc:654] T 2155e8935c0d4607a473779d9e6e2aa9 P e0a8dc515f5440c097b5ad17d5cf216d: Neither blocks nor log segments found. Creating new log.
I20260808 22:30:46.407408 3938 tablet_bootstrap.cc:492] T 2155e8935c0d4607a473779d9e6e2aa9 P 401f2da7203643aeaceb4e1f36f1597f: Bootstrap starting.
I20260808 22:30:46.407728 3940 log.cc:824] T 2155e8935c0d4607a473779d9e6e2aa9 P e0a8dc515f5440c097b5ad17d5cf216d: Log is configured to *not* fsync() on all Append() calls
I20260808 22:30:46.409017 3937 tablet_bootstrap.cc:492] T 2155e8935c0d4607a473779d9e6e2aa9 P 88e13b8cfeb844cf94f46b6ac8f75f81: No bootstrap required, opened a new log
I20260808 22:30:46.409284 3937 ts_tablet_manager.cc:1397] T 2155e8935c0d4607a473779d9e6e2aa9 P 88e13b8cfeb844cf94f46b6ac8f75f81: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 22:30:46.409593 3938 tablet_bootstrap.cc:654] T 2155e8935c0d4607a473779d9e6e2aa9 P 401f2da7203643aeaceb4e1f36f1597f: Neither blocks nor log segments found. Creating new log.
I20260808 22:30:46.410255 3940 tablet_bootstrap.cc:492] T 2155e8935c0d4607a473779d9e6e2aa9 P e0a8dc515f5440c097b5ad17d5cf216d: No bootstrap required, opened a new log
I20260808 22:30:46.410465 3940 ts_tablet_manager.cc:1397] T 2155e8935c0d4607a473779d9e6e2aa9 P e0a8dc515f5440c097b5ad17d5cf216d: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260808 22:30:46.410459 3938 log.cc:824] T 2155e8935c0d4607a473779d9e6e2aa9 P 401f2da7203643aeaceb4e1f36f1597f: Log is configured to *not* fsync() on all Append() calls
I20260808 22:30:46.412436 3938 tablet_bootstrap.cc:492] T 2155e8935c0d4607a473779d9e6e2aa9 P 401f2da7203643aeaceb4e1f36f1597f: No bootstrap required, opened a new log
I20260808 22:30:46.412621 3938 ts_tablet_manager.cc:1397] T 2155e8935c0d4607a473779d9e6e2aa9 P 401f2da7203643aeaceb4e1f36f1597f: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 22:30:46.412957 3937 raft_consensus.cc:348] T 2155e8935c0d4607a473779d9e6e2aa9 P 88e13b8cfeb844cf94f46b6ac8f75f81 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "401f2da7203643aeaceb4e1f36f1597f" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36315 } } peers { permanent_uuid: "88e13b8cfeb844cf94f46b6ac8f75f81" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 33667 } } peers { permanent_uuid: "e0a8dc515f5440c097b5ad17d5cf216d" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 35545 } }
I20260808 22:30:46.413192 3937 raft_consensus.cc:374] T 2155e8935c0d4607a473779d9e6e2aa9 P 88e13b8cfeb844cf94f46b6ac8f75f81 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:30:46.413268 3937 raft_consensus.cc:729] T 2155e8935c0d4607a473779d9e6e2aa9 P 88e13b8cfeb844cf94f46b6ac8f75f81 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88e13b8cfeb844cf94f46b6ac8f75f81, State: Initialized, Role: FOLLOWER
I20260808 22:30:46.413731 3937 consensus_queue.cc:260] T 2155e8935c0d4607a473779d9e6e2aa9 P 88e13b8cfeb844cf94f46b6ac8f75f81 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "401f2da7203643aeaceb4e1f36f1597f" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36315 } } peers { permanent_uuid: "88e13b8cfeb844cf94f46b6ac8f75f81" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 33667 } } peers { permanent_uuid: "e0a8dc515f5440c097b5ad17d5cf216d" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 35545 } }
I20260808 22:30:46.414362 3676 heartbeater.cc:503] Master 127.5.11.254:36889 was elected leader, sending a full tablet report...
I20260808 22:30:46.414296 3940 raft_consensus.cc:348] T 2155e8935c0d4607a473779d9e6e2aa9 P e0a8dc515f5440c097b5ad17d5cf216d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "401f2da7203643aeaceb4e1f36f1597f" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36315 } } peers { permanent_uuid: "88e13b8cfeb844cf94f46b6ac8f75f81" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 33667 } } peers { permanent_uuid: "e0a8dc515f5440c097b5ad17d5cf216d" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 35545 } }
I20260808 22:30:46.414561 3940 raft_consensus.cc:374] T 2155e8935c0d4607a473779d9e6e2aa9 P e0a8dc515f5440c097b5ad17d5cf216d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:30:46.414629 3940 raft_consensus.cc:729] T 2155e8935c0d4607a473779d9e6e2aa9 P e0a8dc515f5440c097b5ad17d5cf216d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0a8dc515f5440c097b5ad17d5cf216d, State: Initialized, Role: FOLLOWER
I20260808 22:30:46.415043 3940 consensus_queue.cc:260] T 2155e8935c0d4607a473779d9e6e2aa9 P e0a8dc515f5440c097b5ad17d5cf216d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "401f2da7203643aeaceb4e1f36f1597f" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36315 } } peers { permanent_uuid: "88e13b8cfeb844cf94f46b6ac8f75f81" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 33667 } } peers { permanent_uuid: "e0a8dc515f5440c097b5ad17d5cf216d" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 35545 } }
I20260808 22:30:46.416062 3940 ts_tablet_manager.cc:1428] T 2155e8935c0d4607a473779d9e6e2aa9 P e0a8dc515f5440c097b5ad17d5cf216d: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 22:30:46.416425 3453 heartbeater.cc:503] Master 127.5.11.254:36889 was elected leader, sending a full tablet report...
I20260808 22:30:46.416646 3938 raft_consensus.cc:348] T 2155e8935c0d4607a473779d9e6e2aa9 P 401f2da7203643aeaceb4e1f36f1597f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "401f2da7203643aeaceb4e1f36f1597f" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36315 } } peers { permanent_uuid: "88e13b8cfeb844cf94f46b6ac8f75f81" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 33667 } } peers { permanent_uuid: "e0a8dc515f5440c097b5ad17d5cf216d" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 35545 } }
I20260808 22:30:46.416815 3938 raft_consensus.cc:374] T 2155e8935c0d4607a473779d9e6e2aa9 P 401f2da7203643aeaceb4e1f36f1597f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:30:46.416852 3938 raft_consensus.cc:729] T 2155e8935c0d4607a473779d9e6e2aa9 P 401f2da7203643aeaceb4e1f36f1597f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 401f2da7203643aeaceb4e1f36f1597f, State: Initialized, Role: FOLLOWER
I20260808 22:30:46.417201 3938 consensus_queue.cc:260] T 2155e8935c0d4607a473779d9e6e2aa9 P 401f2da7203643aeaceb4e1f36f1597f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "401f2da7203643aeaceb4e1f36f1597f" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36315 } } peers { permanent_uuid: "88e13b8cfeb844cf94f46b6ac8f75f81" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 33667 } } peers { permanent_uuid: "e0a8dc515f5440c097b5ad17d5cf216d" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 35545 } }
I20260808 22:30:46.417847 3904 heartbeater.cc:503] Master 127.5.11.254:36889 was elected leader, sending a full tablet report...
I20260808 22:30:46.417872 3937 ts_tablet_manager.cc:1428] T 2155e8935c0d4607a473779d9e6e2aa9 P 88e13b8cfeb844cf94f46b6ac8f75f81: Time spent starting tablet: real 0.009s user 0.006s sys 0.000s
I20260808 22:30:46.418184 3938 ts_tablet_manager.cc:1428] T 2155e8935c0d4607a473779d9e6e2aa9 P 401f2da7203643aeaceb4e1f36f1597f: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260808 22:30:46.419977 3954 raft_consensus.cc:482] T 2155e8935c0d4607a473779d9e6e2aa9 P e0a8dc515f5440c097b5ad17d5cf216d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:30:46.420253 3954 raft_consensus.cc:504] T 2155e8935c0d4607a473779d9e6e2aa9 P e0a8dc515f5440c097b5ad17d5cf216d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "401f2da7203643aeaceb4e1f36f1597f" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36315 } } peers { permanent_uuid: "88e13b8cfeb844cf94f46b6ac8f75f81" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 33667 } } peers { permanent_uuid: "e0a8dc515f5440c097b5ad17d5cf216d" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 35545 } }
I20260808 22:30:46.421428 3954 leader_election.cc:288] T 2155e8935c0d4607a473779d9e6e2aa9 P e0a8dc515f5440c097b5ad17d5cf216d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 401f2da7203643aeaceb4e1f36f1597f (127.5.11.195:36315), 88e13b8cfeb844cf94f46b6ac8f75f81 (127.5.11.194:33667)
I20260808 22:30:46.425239 3455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:46.421628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:36315 (local address 127.5.11.193:48813)
0808 22:30:46.421726 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:46.421744 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:46.421825 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:46.422226 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:46.422230 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:46.422247 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:30:46.422381 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:46.422389 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:46.423151 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:46.423156 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:46.424069 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:46.424080 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:46.424933 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:46.424935 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:46.425012 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:46.425033 (+ 21us) client_negotiation.cc:769] Sending connection context
0808 22:30:46.425095 (+ 62us) client_negotiation.cc:240] Negotiation successful
0808 22:30:46.425147 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260808 22:30:46.425330 3929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:46.421901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:48813 (local address 127.5.11.195:36315)
0808 22:30:46.422003 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:46.422009 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:46.422024 (+ 15us) server_negotiation.cc:407] Connection header received
0808 22:30:46.422075 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:46.422081 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:46.422127 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:46.422201 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:30:46.422501 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:46.423044 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:46.424201 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:46.424820 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:46.424940 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:46.424988 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:46.425126 (+ 138us) server_negotiation.cc:299] Negotiation successful
0808 22:30:46.425205 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 22:30:46.425670 3935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:46.421688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:43707 (local address 127.5.11.194:33667)
0808 22:30:46.421811 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:46.421818 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:46.422103 (+ 285us) server_negotiation.cc:407] Connection header received
0808 22:30:46.422211 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:46.422214 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:46.422258 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:46.422340 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:30:46.422641 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:46.423138 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:46.424400 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:46.424994 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:46.425077 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:46.425125 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:46.425489 (+ 364us) server_negotiation.cc:299] Negotiation successful
0808 22:30:46.425560 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 22:30:46.425871 3827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2155e8935c0d4607a473779d9e6e2aa9" candidate_uuid: "e0a8dc515f5440c097b5ad17d5cf216d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "401f2da7203643aeaceb4e1f36f1597f" is_pre_election: true
I20260808 22:30:46.426024 3958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:46.421628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:33667 (local address 127.5.11.193:43707)
0808 22:30:46.422003 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:46.422017 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:46.422117 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:46.422378 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:46.422381 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:46.422395 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:30:46.422533 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:46.422540 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:46.423250 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:46.423252 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:46.424258 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:46.424268 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:46.425221 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:46.425225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:46.425323 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:46.425356 (+ 33us) client_negotiation.cc:769] Sending connection context
0808 22:30:46.425830 (+ 474us) client_negotiation.cc:240] Negotiation successful
0808 22:30:46.425903 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":38,"thread_start_us":109,"threads_started":1}
I20260808 22:30:46.426147 3827 raft_consensus.cc:2454] T 2155e8935c0d4607a473779d9e6e2aa9 P 401f2da7203643aeaceb4e1f36f1597f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0a8dc515f5440c097b5ad17d5cf216d in term 0.
I20260808 22:30:46.426599 3238 leader_election.cc:302] T 2155e8935c0d4607a473779d9e6e2aa9 P e0a8dc515f5440c097b5ad17d5cf216d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 401f2da7203643aeaceb4e1f36f1597f, e0a8dc515f5440c097b5ad17d5cf216d; no voters:
I20260808 22:30:46.426640 3601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2155e8935c0d4607a473779d9e6e2aa9" candidate_uuid: "e0a8dc515f5440c097b5ad17d5cf216d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88e13b8cfeb844cf94f46b6ac8f75f81" is_pre_election: true
I20260808 22:30:46.426885 3601 raft_consensus.cc:2454] T 2155e8935c0d4607a473779d9e6e2aa9 P 88e13b8cfeb844cf94f46b6ac8f75f81 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e0a8dc515f5440c097b5ad17d5cf216d in term 0.
I20260808 22:30:46.426877 3954 raft_consensus.cc:2781] T 2155e8935c0d4607a473779d9e6e2aa9 P e0a8dc515f5440c097b5ad17d5cf216d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:30:46.426973 3954 raft_consensus.cc:482] T 2155e8935c0d4607a473779d9e6e2aa9 P e0a8dc515f5440c097b5ad17d5cf216d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:30:46.427047 3954 raft_consensus.cc:3037] T 2155e8935c0d4607a473779d9e6e2aa9 P e0a8dc515f5440c097b5ad17d5cf216d [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:30:46.428097 3954 raft_consensus.cc:504] T 2155e8935c0d4607a473779d9e6e2aa9 P e0a8dc515f5440c097b5ad17d5cf216d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "401f2da7203643aeaceb4e1f36f1597f" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36315 } } peers { permanent_uuid: "88e13b8cfeb844cf94f46b6ac8f75f81" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 33667 } } peers { permanent_uuid: "e0a8dc515f5440c097b5ad17d5cf216d" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 35545 } }
I20260808 22:30:46.428452 3954 leader_election.cc:288] T 2155e8935c0d4607a473779d9e6e2aa9 P e0a8dc515f5440c097b5ad17d5cf216d [CANDIDATE]: Term 1 election: Requested vote from peers 401f2da7203643aeaceb4e1f36f1597f (127.5.11.195:36315), 88e13b8cfeb844cf94f46b6ac8f75f81 (127.5.11.194:33667)
I20260808 22:30:46.428745 3827 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2155e8935c0d4607a473779d9e6e2aa9" candidate_uuid: "e0a8dc515f5440c097b5ad17d5cf216d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "401f2da7203643aeaceb4e1f36f1597f"
I20260808 22:30:46.428783 3601 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2155e8935c0d4607a473779d9e6e2aa9" candidate_uuid: "e0a8dc515f5440c097b5ad17d5cf216d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88e13b8cfeb844cf94f46b6ac8f75f81"
I20260808 22:30:46.428850 3827 raft_consensus.cc:3037] T 2155e8935c0d4607a473779d9e6e2aa9 P 401f2da7203643aeaceb4e1f36f1597f [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:30:46.428952 3601 raft_consensus.cc:3037] T 2155e8935c0d4607a473779d9e6e2aa9 P 88e13b8cfeb844cf94f46b6ac8f75f81 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:30:46.429812 3827 raft_consensus.cc:2454] T 2155e8935c0d4607a473779d9e6e2aa9 P 401f2da7203643aeaceb4e1f36f1597f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a8dc515f5440c097b5ad17d5cf216d in term 1.
I20260808 22:30:46.429993 3601 raft_consensus.cc:2454] T 2155e8935c0d4607a473779d9e6e2aa9 P 88e13b8cfeb844cf94f46b6ac8f75f81 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e0a8dc515f5440c097b5ad17d5cf216d in term 1.
I20260808 22:30:46.430157 3238 leader_election.cc:302] T 2155e8935c0d4607a473779d9e6e2aa9 P e0a8dc515f5440c097b5ad17d5cf216d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 401f2da7203643aeaceb4e1f36f1597f, e0a8dc515f5440c097b5ad17d5cf216d; no voters:
I20260808 22:30:46.430351 3954 raft_consensus.cc:2781] T 2155e8935c0d4607a473779d9e6e2aa9 P e0a8dc515f5440c097b5ad17d5cf216d [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:30:46.430581 3954 raft_consensus.cc:686] T 2155e8935c0d4607a473779d9e6e2aa9 P e0a8dc515f5440c097b5ad17d5cf216d [term 1 LEADER]: Becoming Leader. State: Replica: e0a8dc515f5440c097b5ad17d5cf216d, State: Running, Role: LEADER
I20260808 22:30:46.430869 3954 consensus_queue.cc:237] T 2155e8935c0d4607a473779d9e6e2aa9 P e0a8dc515f5440c097b5ad17d5cf216d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "401f2da7203643aeaceb4e1f36f1597f" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36315 } } peers { permanent_uuid: "88e13b8cfeb844cf94f46b6ac8f75f81" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 33667 } } peers { permanent_uuid: "e0a8dc515f5440c097b5ad17d5cf216d" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 35545 } }
I20260808 22:30:46.433955 3123 catalog_manager.cc:5495] T 2155e8935c0d4607a473779d9e6e2aa9 P e0a8dc515f5440c097b5ad17d5cf216d reported cstate change: term changed from 0 to 1, leader changed from <none> to e0a8dc515f5440c097b5ad17d5cf216d (127.5.11.193). New cstate: current_term: 1 leader_uuid: "e0a8dc515f5440c097b5ad17d5cf216d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "401f2da7203643aeaceb4e1f36f1597f" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36315 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88e13b8cfeb844cf94f46b6ac8f75f81" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 33667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e0a8dc515f5440c097b5ad17d5cf216d" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 35545 } health_report { overall_health: HEALTHY } } }
I20260808 22:30:46.454977 3936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:46.451677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55178 (local address 127.5.11.193:35545)
0808 22:30:46.451825 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:46.451829 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:46.451841 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:30:46.451969 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:46.451972 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:46.452022 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:46.452108 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:30:46.452317 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:46.452777 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:46.453593 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:46.453851 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:46.453947 (+ 96us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:46.454237 (+ 290us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:30:46.454604 (+ 367us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:30:46.454709 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:46.454833 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 22:30:46.454881 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 22:30:46.460465 3827 raft_consensus.cc:1261] T 2155e8935c0d4607a473779d9e6e2aa9 P 401f2da7203643aeaceb4e1f36f1597f [term 1 FOLLOWER]: Refusing update from remote peer e0a8dc515f5440c097b5ad17d5cf216d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:30:46.460464 3601 raft_consensus.cc:1261] T 2155e8935c0d4607a473779d9e6e2aa9 P 88e13b8cfeb844cf94f46b6ac8f75f81 [term 1 FOLLOWER]: Refusing update from remote peer e0a8dc515f5440c097b5ad17d5cf216d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:30:46.461113 3960 consensus_queue.cc:1035] T 2155e8935c0d4607a473779d9e6e2aa9 P e0a8dc515f5440c097b5ad17d5cf216d [LEADER]: Connected to new peer: Peer: permanent_uuid: "88e13b8cfeb844cf94f46b6ac8f75f81" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 33667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:30:46.461306 3954 consensus_queue.cc:1035] T 2155e8935c0d4607a473779d9e6e2aa9 P e0a8dc515f5440c097b5ad17d5cf216d [LEADER]: Connected to new peer: Peer: permanent_uuid: "401f2da7203643aeaceb4e1f36f1597f" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 36315 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:30:46.469393 3963 mvcc.cc:205] Tried to move back new op lower bound from 7316390897495203840 to 7316390897383194624. Current Snapshot: MvccSnapshot[applied={T|T < 7316390897495203840}]
I20260808 22:30:46.470422 3964 mvcc.cc:205] Tried to move back new op lower bound from 7316390897495203840 to 7316390897383194624. Current Snapshot: MvccSnapshot[applied={T|T < 7316390897495203840}]
I20260808 22:30:46.471797 3967 mvcc.cc:205] Tried to move back new op lower bound from 7316390897495203840 to 7316390897383194624. Current Snapshot: MvccSnapshot[applied={T|T < 7316390897495203840}]
I20260808 22:30:46.477878 3117 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33488:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:30:46.478010 3117 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33488:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:30:46.480278 3117 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 063565472c8340d28d039ff28b8a4ee7: Sending DeleteTablet for 3 replicas of tablet 2155e8935c0d4607a473779d9e6e2aa9
I20260808 22:30:46.481019 3794 tablet_service.cc:1514] Processing DeleteTablet for tablet 2155e8935c0d4607a473779d9e6e2aa9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:30:46 UTC) from {username='slave'} at 127.0.0.1:41220
I20260808 22:30:46.481266 3319 tablet_service.cc:1514] Processing DeleteTablet for tablet 2155e8935c0d4607a473779d9e6e2aa9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:30:46 UTC) from {username='slave'} at 127.0.0.1:55170
I20260808 22:30:46.481488 3567 tablet_service.cc:1514] Processing DeleteTablet for tablet 2155e8935c0d4607a473779d9e6e2aa9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:30:46 UTC) from {username='slave'} at 127.0.0.1:49194
I20260808 22:30:46.481711 3978 tablet_replica.cc:330] T 2155e8935c0d4607a473779d9e6e2aa9 P e0a8dc515f5440c097b5ad17d5cf216d: stopping tablet replica
I20260808 22:30:46.481978 3979 tablet_replica.cc:330] T 2155e8935c0d4607a473779d9e6e2aa9 P 88e13b8cfeb844cf94f46b6ac8f75f81: stopping tablet replica
I20260808 22:30:46.482096 3977 tablet_replica.cc:330] T 2155e8935c0d4607a473779d9e6e2aa9 P 401f2da7203643aeaceb4e1f36f1597f: stopping tablet replica
I20260808 22:30:46.482375 3979 raft_consensus.cc:2229] T 2155e8935c0d4607a473779d9e6e2aa9 P 88e13b8cfeb844cf94f46b6ac8f75f81 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:30:46.482555 3977 raft_consensus.cc:2229] T 2155e8935c0d4607a473779d9e6e2aa9 P 401f2da7203643aeaceb4e1f36f1597f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:30:46.482600 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 3220
I20260808 22:30:46.482846 3977 raft_consensus.cc:2258] T 2155e8935c0d4607a473779d9e6e2aa9 P 401f2da7203643aeaceb4e1f36f1597f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:30:46.483357 3979 raft_consensus.cc:2258] T 2155e8935c0d4607a473779d9e6e2aa9 P 88e13b8cfeb844cf94f46b6ac8f75f81 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:30:46.484306 3977 ts_tablet_manager.cc:1905] T 2155e8935c0d4607a473779d9e6e2aa9 P 401f2da7203643aeaceb4e1f36f1597f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:30:46.487705 3977 ts_tablet_manager.cc:1918] T 2155e8935c0d4607a473779d9e6e2aa9 P 401f2da7203643aeaceb4e1f36f1597f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:30:46.487835 3977 log.cc:1196] T 2155e8935c0d4607a473779d9e6e2aa9 P 401f2da7203643aeaceb4e1f36f1597f: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-2/wal/wals/2155e8935c0d4607a473779d9e6e2aa9
I20260808 22:30:46.488188 3977 ts_tablet_manager.cc:1939] T 2155e8935c0d4607a473779d9e6e2aa9 P 401f2da7203643aeaceb4e1f36f1597f: Deleting consensus metadata
I20260808 22:30:46.489018 3979 ts_tablet_manager.cc:1905] T 2155e8935c0d4607a473779d9e6e2aa9 P 88e13b8cfeb844cf94f46b6ac8f75f81: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:30:46.489349 3093 catalog_manager.cc:4841] TS 401f2da7203643aeaceb4e1f36f1597f (127.5.11.195:36315): tablet 2155e8935c0d4607a473779d9e6e2aa9 (table test-table [id=b7a3d0ea51604c52ab9338180968ab59]) successfully deleted
I20260808 22:30:46.492318 3979 ts_tablet_manager.cc:1918] T 2155e8935c0d4607a473779d9e6e2aa9 P 88e13b8cfeb844cf94f46b6ac8f75f81: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:30:46.492429 3979 log.cc:1196] T 2155e8935c0d4607a473779d9e6e2aa9 P 88e13b8cfeb844cf94f46b6ac8f75f81: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786227991988852-5167-0/minicluster-data/ts-1/wal/wals/2155e8935c0d4607a473779d9e6e2aa9
I20260808 22:30:46.492786 3979 ts_tablet_manager.cc:1939] T 2155e8935c0d4607a473779d9e6e2aa9 P 88e13b8cfeb844cf94f46b6ac8f75f81: Deleting consensus metadata
I20260808 22:30:46.493952 3099 catalog_manager.cc:4841] TS 88e13b8cfeb844cf94f46b6ac8f75f81 (127.5.11.194:33667): tablet 2155e8935c0d4607a473779d9e6e2aa9 (table test-table [id=b7a3d0ea51604c52ab9338180968ab59]) successfully deleted
W20260808 22:30:46.497097 3094 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.11.193:35545 (error 108)
I20260808 22:30:46.497867 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 3456
I20260808 22:30:46.498171 3932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:46.497830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:35545 (local address 127.0.0.1:55182)
0808 22:30:46.497998 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:46.498077 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.193:35545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260808 22:30:46.512179 3094 catalog_manager.cc:4584] TS e0a8dc515f5440c097b5ad17d5cf216d (127.5.11.193:35545): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2155e8935c0d4607a473779d9e6e2aa9: Network error: Client connection negotiation failed: client connection to 127.5.11.193:35545: connect: Connection refused (error 111)
I20260808 22:30:46.518292 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 3681
I20260808 22:30:46.531081 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 3070
2026-08-08T22:30:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1024 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:30:46 dist-test-slave-7f01 krb5kdc[3984](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:30:46 dist-test-slave-7f01 krb5kdc[3984](info): set up 2 sockets
Aug 08 22:30:46 dist-test-slave-7f01 krb5kdc[3984](info): commencing operation
krb5kdc: starting...
W20260808 22:30:47.777185 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.219s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:30:47 dist-test-slave-7f01 krb5kdc[3984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:30:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:30:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.11.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
I20260808 22:30:47.906236 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:43289
--webserver_interface=127.5.11.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:46391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:43289
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:30:48.018059 4009 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:30:48.018347 4009 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:30:48.018445 4009 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:30:48.022377 4009 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:30:48.022465 4009 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:30:48.022504 4009 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:30:48.022563 4009 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:30:48.022603 4009 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:30:48.027629 4009 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:46391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:43289
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:43289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4009
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:30:48.028636 4009 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:30:48.029615 4009 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:30:48.037958 4020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:48.037961 4017 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:48.037963 4023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:48.038229 4009 server_base.cc:1034] running on GCE node
I20260808 22:30:48.038873 4009 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:30:48.040000 4009 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:30:48.041200 4009 hybrid_clock.cc:648] HybridClock initialized: now 1786228248041182 us; error 30 us; skew 500 ppm
Aug 08 22:30:48 dist-test-slave-7f01 krb5kdc[3984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228248, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:30:48.043908 4009 init.cc:377] Logged in from keytab as kudu/127.5.11.254@KRBTEST.COM (short username kudu)
I20260808 22:30:48.044754 4009 webserver.cc:460] Webserver started at http://127.5.11.254:39827/ using document root <none> and password file <none>
I20260808 22:30:48.045312 4009 fs_manager.cc:359] Metadata directory not provided
I20260808 22:30:48.045408 4009 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:30:48.045642 4009 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:30:48.047616 4009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "8c15714816594854a5f22bb2e6e2e4f2"
format_stamp: "Formatted at 2026-08-08 22:30:48 on dist-test-slave-7f01"
server_key: "50689095c0b3f4e5e9c149e82198c186"
server_key_iv: "aff0c3f002eb28910e443af7ae89e250"
server_key_version: "encryptionkey@0"
I20260808 22:30:48.048244 4009 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "8c15714816594854a5f22bb2e6e2e4f2"
format_stamp: "Formatted at 2026-08-08 22:30:48 on dist-test-slave-7f01"
server_key: "50689095c0b3f4e5e9c149e82198c186"
server_key_iv: "aff0c3f002eb28910e443af7ae89e250"
server_key_version: "encryptionkey@0"
I20260808 22:30:48.051931 4009 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:30:48.054366 4033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:48.055619 4009 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:30:48.055838 4009 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "8c15714816594854a5f22bb2e6e2e4f2"
format_stamp: "Formatted at 2026-08-08 22:30:48 on dist-test-slave-7f01"
server_key: "50689095c0b3f4e5e9c149e82198c186"
server_key_iv: "aff0c3f002eb28910e443af7ae89e250"
server_key_version: "encryptionkey@0"
I20260808 22:30:48.055970 4009 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:30:48.076092 4009 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:30:48.078979 4009 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:30:48.079195 4009 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:30:48.086565 4009 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:43289
I20260808 22:30:48.086615 4127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:43289 every 8 connection(s)
I20260808 22:30:48.087649 4009 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:30:48.090440 4128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:30:48.092628 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 4009
I20260808 22:30:48.092797 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:30:48.093094 5167 external_mini_cluster.cc:1428] Setting key 7a42babfea99decfc3eb63c20bb2ebac
I20260808 22:30:48.096697 4128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c15714816594854a5f22bb2e6e2e4f2: Bootstrap starting.
I20260808 22:30:48.099309 4128 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c15714816594854a5f22bb2e6e2e4f2: Neither blocks nor log segments found. Creating new log.
I20260808 22:30:48.100201 4128 log.cc:824] T 00000000000000000000000000000000 P 8c15714816594854a5f22bb2e6e2e4f2: Log is configured to *not* fsync() on all Append() calls
I20260808 22:30:48.102219 4128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c15714816594854a5f22bb2e6e2e4f2: No bootstrap required, opened a new log
Aug 08 22:30:48 dist-test-slave-7f01 krb5kdc[3984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228247, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:30:48.105438 4128 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c15714816594854a5f22bb2e6e2e4f2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c15714816594854a5f22bb2e6e2e4f2" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 43289 } }
I20260808 22:30:48.105700 4128 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8c15714816594854a5f22bb2e6e2e4f2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:30:48.105782 4128 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c15714816594854a5f22bb2e6e2e4f2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c15714816594854a5f22bb2e6e2e4f2, State: Initialized, Role: FOLLOWER
I20260808 22:30:48.106335 4128 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c15714816594854a5f22bb2e6e2e4f2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c15714816594854a5f22bb2e6e2e4f2" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 43289 } }
I20260808 22:30:48.106529 4128 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8c15714816594854a5f22bb2e6e2e4f2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:30:48.106616 4128 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8c15714816594854a5f22bb2e6e2e4f2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:30:48.106736 4128 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c15714816594854a5f22bb2e6e2e4f2 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:30:48.107919 4128 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8c15714816594854a5f22bb2e6e2e4f2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c15714816594854a5f22bb2e6e2e4f2" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 43289 } }
I20260808 22:30:48.108286 4128 leader_election.cc:302] T 00000000000000000000000000000000 P 8c15714816594854a5f22bb2e6e2e4f2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8c15714816594854a5f22bb2e6e2e4f2; no voters:
I20260808 22:30:48.108657 4128 leader_election.cc:288] T 00000000000000000000000000000000 P 8c15714816594854a5f22bb2e6e2e4f2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:30:48.108749 4135 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8c15714816594854a5f22bb2e6e2e4f2 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:30:48.108969 4135 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8c15714816594854a5f22bb2e6e2e4f2 [term 1 LEADER]: Becoming Leader. State: Replica: 8c15714816594854a5f22bb2e6e2e4f2, State: Running, Role: LEADER
I20260808 22:30:48.109002 4133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:48.094520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46838 (local address 127.5.11.254:43289)
0808 22:30:48.094935 (+ 415us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:48.094953 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:48.095010 (+ 57us) server_negotiation.cc:407] Connection header received
0808 22:30:48.095855 (+ 845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:48.095880 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:48.096288 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:48.096644 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:48.097415 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:48.098901 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:48.099793 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:48.100488 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:48.100661 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:48.103417 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:48.103422 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:48.103435 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:30:48.103460 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:48.105761 (+ 2301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:48.106273 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:48.106277 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:48.106282 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:48.106394 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:48.106724 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:48.106728 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:48.106730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:48.107208 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:48.107353 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:48.107691 (+ 338us) server_negotiation.cc:299] Negotiation successful
0808 22:30:48.108070 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":98,"threads_started":1}
I20260808 22:30:48.109666 4135 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8c15714816594854a5f22bb2e6e2e4f2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c15714816594854a5f22bb2e6e2e4f2" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 43289 } }
I20260808 22:30:48.109959 4128 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c15714816594854a5f22bb2e6e2e4f2 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:30:48.111370 4138 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c15714816594854a5f22bb2e6e2e4f2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8c15714816594854a5f22bb2e6e2e4f2. Latest consensus state: current_term: 1 leader_uuid: "8c15714816594854a5f22bb2e6e2e4f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c15714816594854a5f22bb2e6e2e4f2" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 43289 } } }
I20260808 22:30:48.111611 4137 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c15714816594854a5f22bb2e6e2e4f2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8c15714816594854a5f22bb2e6e2e4f2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8c15714816594854a5f22bb2e6e2e4f2" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 43289 } } }
I20260808 22:30:48.111773 4138 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c15714816594854a5f22bb2e6e2e4f2 [sys.catalog]: This master's current role is: LEADER
I20260808 22:30:48.111811 4137 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c15714816594854a5f22bb2e6e2e4f2 [sys.catalog]: This master's current role is: LEADER
W20260808 22:30:48.114681 4160 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8c15714816594854a5f22bb2e6e2e4f2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:30:48.114754 4160 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:30:48.114840 4158 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:30:48.115720 4158 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:30:48.120944 4158 catalog_manager.cc:1347] Generated new cluster ID: 2335cc4b801645a18bbad4ba6e7a685d
I20260808 22:30:48.121008 4158 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:30:48.137262 4158 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:30:48.138419 4158 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:30:48.150947 4158 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8c15714816594854a5f22bb2e6e2e4f2: Generated new TSK 0
I20260808 22:30:48.151567 4158 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.5.11.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab.
Entry for principal kudu/127.5.11.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab.
I20260808 22:30:48.189180 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.193:0
--local_ip_for_outbound_sockets=127.5.11.193
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:43289
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:46391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:30:48.301635 4169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:30:48.301913 4169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:30:48.301992 4169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:30:48.305935 4169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:30:48.306020 4169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:30:48.306109 4169 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.193
I20260808 22:30:48.311470 4169 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:46391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.193.keytab
--principal=kudu/127.5.11.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:43289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.193
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:30:48.312546 4169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:30:48.313562 4169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:30:48.323088 4175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:48.323124 4176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:48.323238 4179 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:48.323125 4169 server_base.cc:1034] running on GCE node
I20260808 22:30:48.323680 4169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:30:48.324379 4169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:30:48.325577 4169 hybrid_clock.cc:648] HybridClock initialized: now 1786228248325545 us; error 44 us; skew 500 ppm
Aug 08 22:30:48 dist-test-slave-7f01 krb5kdc[3984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228248, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:30:48.328413 4169 init.cc:377] Logged in from keytab as kudu/127.5.11.193@KRBTEST.COM (short username kudu)
I20260808 22:30:48.329226 4169 webserver.cc:460] Webserver started at http://127.5.11.193:46283/ using document root <none> and password file <none>
I20260808 22:30:48.329789 4169 fs_manager.cc:359] Metadata directory not provided
I20260808 22:30:48.329879 4169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:30:48.330183 4169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:30:48.332271 4169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-0/data/instance:
uuid: "36345df1409c4f50bc02e3bdfa020b5d"
format_stamp: "Formatted at 2026-08-08 22:30:48 on dist-test-slave-7f01"
server_key: "2c8aee4dac702abfc6b566a9e8a8405c"
server_key_iv: "48eb3d179c665ccbd12034c0896ddcfa"
server_key_version: "encryptionkey@0"
I20260808 22:30:48.332882 4169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance:
uuid: "36345df1409c4f50bc02e3bdfa020b5d"
format_stamp: "Formatted at 2026-08-08 22:30:48 on dist-test-slave-7f01"
server_key: "2c8aee4dac702abfc6b566a9e8a8405c"
server_key_iv: "48eb3d179c665ccbd12034c0896ddcfa"
server_key_version: "encryptionkey@0"
I20260808 22:30:48.336567 4169 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:30:48.338986 4191 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:48.340212 4169 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:30:48.340348 4169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-0/wal
uuid: "36345df1409c4f50bc02e3bdfa020b5d"
format_stamp: "Formatted at 2026-08-08 22:30:48 on dist-test-slave-7f01"
server_key: "2c8aee4dac702abfc6b566a9e8a8405c"
server_key_iv: "48eb3d179c665ccbd12034c0896ddcfa"
server_key_version: "encryptionkey@0"
I20260808 22:30:48.340456 4169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:30:48.357347 4169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:30:48.360332 4169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:30:48.360548 4169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:30:48.361138 4169 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:30:48.361953 4169 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:30:48.362001 4169 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:48.362066 4169 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:30:48.362098 4169 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:48.370576 4169 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.193:41603
I20260808 22:30:48.370640 4374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.193:41603 every 8 connection(s)
I20260808 22:30:48.371464 4169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
I20260808 22:30:48.375662 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 4169
I20260808 22:30:48.375844 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance
I20260808 22:30:48.376135 5167 external_mini_cluster.cc:1428] Setting key 06a0c467865a0095ec9f4c83c2826a76
Aug 08 22:30:48 dist-test-slave-7f01 krb5kdc[3984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228248, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.193@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:30:48.385679 4133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:48.373211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:59327 (local address 127.5.11.254:43289)
0808 22:30:48.373369 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:48.373374 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:48.374160 (+ 786us) server_negotiation.cc:407] Connection header received
0808 22:30:48.375445 (+ 1285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:48.375450 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:48.375513 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:48.375619 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:48.376932 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:48.377585 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:48.378708 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:48.379120 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:48.379244 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:48.381986 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:48.381990 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:48.381993 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:30:48.382020 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:48.383461 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:48.384085 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:48.384088 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:48.384090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:48.384185 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:48.384539 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:48.384542 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:48.384544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:48.384738 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:48.384851 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:48.385359 (+ 508us) server_negotiation.cc:299] Negotiation successful
0808 22:30:48.385501 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 22:30:48.386554 4377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:48.373580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43289 (local address 127.5.11.193:59327)
0808 22:30:48.374015 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:48.374057 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:48.375233 (+ 1176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:48.375814 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:48.375825 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:48.376279 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:30:48.376767 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:48.376780 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:48.377715 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:48.377718 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:48.378577 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:48.378587 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:48.379267 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:48.379270 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:48.379404 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:48.380062 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:30:48.380086 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:30:48.381820 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:30:48.383590 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:30:48.383596 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:30:48.383610 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:30:48.383971 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:30:48.384280 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:30:48.384283 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:30:48.384286 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:30:48.384434 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:30:48.384851 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:30:48.384858 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:30:48.385114 (+ 256us) client_negotiation.cc:769] Sending connection context
0808 22:30:48.385300 (+ 186us) client_negotiation.cc:240] Negotiation successful
0808 22:30:48.385623 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":110,"threads_started":1}
WARNING: no policy specified for kudu/127.5.11.194@KRBTEST.COM; defaulting to no policy
I20260808 22:30:48.388388 4375 heartbeater.cc:347] Connected to a master server at 127.5.11.254:43289
I20260808 22:30:48.388646 4375 heartbeater.cc:464] Registering TS with master...
I20260808 22:30:48.389159 4375 heartbeater.cc:511] Master 127.5.11.254:43289 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.194@KRBTEST.COM" created.
I20260808 22:30:48.390766 4073 ts_manager.cc:194] Registered new tserver with Master: 36345df1409c4f50bc02e3bdfa020b5d (127.5.11.193:41603)
I20260808 22:30:48.391991 4073 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.193@KRBTEST.COM'} at 127.5.11.193:59327
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab.
Entry for principal kudu/127.5.11.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab.
I20260808 22:30:48.406213 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.194:0
--local_ip_for_outbound_sockets=127.5.11.194
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:43289
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:46391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:30:48.518230 4380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:30:48.518477 4380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:30:48.518519 4380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:30:48.522527 4380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:30:48.522578 4380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:30:48.522634 4380 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.194
I20260808 22:30:48.527660 4380 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:46391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.194.keytab
--principal=kudu/127.5.11.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:43289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.194
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:30:48.528668 4380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:30:48.529634 4380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:30:48.539017 4397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:48.539031 4395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:48.539179 4380 server_base.cc:1034] running on GCE node
W20260808 22:30:48.539048 4393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:48.539688 4380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:30:48.540315 4380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:30:48.541477 4380 hybrid_clock.cc:648] HybridClock initialized: now 1786228248541461 us; error 24 us; skew 500 ppm
Aug 08 22:30:48 dist-test-slave-7f01 krb5kdc[3984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228248, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:30:48.544425 4380 init.cc:377] Logged in from keytab as kudu/127.5.11.194@KRBTEST.COM (short username kudu)
I20260808 22:30:48.545286 4380 webserver.cc:460] Webserver started at http://127.5.11.194:44879/ using document root <none> and password file <none>
I20260808 22:30:48.545840 4380 fs_manager.cc:359] Metadata directory not provided
I20260808 22:30:48.545930 4380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:30:48.546162 4380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:30:48.548203 4380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-1/data/instance:
uuid: "64cd91397ca54ea59ef8324dd977cd80"
format_stamp: "Formatted at 2026-08-08 22:30:48 on dist-test-slave-7f01"
server_key: "6a06592dcea84e778edfc9f610f87ea6"
server_key_iv: "5bdef2876e1f22aa7a387863825d5494"
server_key_version: "encryptionkey@0"
I20260808 22:30:48.548784 4380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance:
uuid: "64cd91397ca54ea59ef8324dd977cd80"
format_stamp: "Formatted at 2026-08-08 22:30:48 on dist-test-slave-7f01"
server_key: "6a06592dcea84e778edfc9f610f87ea6"
server_key_iv: "5bdef2876e1f22aa7a387863825d5494"
server_key_version: "encryptionkey@0"
I20260808 22:30:48.552383 4380 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:30:48.554775 4412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:48.556031 4380 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:30:48.556170 4380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-1/wal
uuid: "64cd91397ca54ea59ef8324dd977cd80"
format_stamp: "Formatted at 2026-08-08 22:30:48 on dist-test-slave-7f01"
server_key: "6a06592dcea84e778edfc9f610f87ea6"
server_key_iv: "5bdef2876e1f22aa7a387863825d5494"
server_key_version: "encryptionkey@0"
I20260808 22:30:48.556280 4380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:30:48.592404 4380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:30:48.595961 4380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:30:48.596186 4380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:30:48.596838 4380 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:30:48.597703 4380 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:30:48.597774 4380 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:48.597848 4380 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:30:48.597900 4380 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:48.607036 4380 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.194:41845
I20260808 22:30:48.607067 4607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.194:41845 every 8 connection(s)
I20260808 22:30:48.608009 4380 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
I20260808 22:30:48.613209 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 4380
I20260808 22:30:48.613349 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance
I20260808 22:30:48.613655 5167 external_mini_cluster.cc:1428] Setting key 402c7307e482645da4f5e3dc3ad2548c
Aug 08 22:30:48 dist-test-slave-7f01 krb5kdc[3984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228248, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.194@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:30:48.624075 4133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:48.609912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:36531 (local address 127.5.11.254:43289)
0808 22:30:48.610068 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:48.610073 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:48.611193 (+ 1120us) server_negotiation.cc:407] Connection header received
0808 22:30:48.612689 (+ 1496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:48.612692 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:48.612742 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:48.612818 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:48.614432 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:48.615189 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:48.616920 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:48.617141 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:48.617227 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:48.620182 (+ 2955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:48.620186 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:48.620188 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:30:48.620212 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:48.621658 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:48.622348 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:48.622353 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:48.622355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:48.622452 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:48.622866 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:48.622868 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:48.622869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:48.623048 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:48.623120 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:48.623839 (+ 719us) server_negotiation.cc:299] Negotiation successful
0808 22:30:48.623942 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 22:30:48.625253 4611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:48.610359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43289 (local address 127.5.11.194:36531)
0808 22:30:48.611017 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:48.611076 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:48.612467 (+ 1391us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:48.613039 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:48.613051 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:48.613601 (+ 550us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:30:48.614258 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:48.614279 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:48.615318 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:48.615321 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:48.616800 (+ 1479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:48.616816 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:48.617296 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:48.617303 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:48.617553 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:48.618165 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:30:48.618181 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:30:48.620020 (+ 1839us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:30:48.621787 (+ 1767us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:30:48.621796 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:30:48.621814 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:30:48.622220 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:30:48.622553 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:30:48.622558 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:30:48.622562 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:30:48.622760 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:30:48.623155 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:30:48.623162 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:30:48.623507 (+ 345us) client_negotiation.cc:769] Sending connection context
0808 22:30:48.623754 (+ 247us) client_negotiation.cc:240] Negotiation successful
0808 22:30:48.624115 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":174,"threads_started":1}
WARNING: no policy specified for kudu/127.5.11.195@KRBTEST.COM; defaulting to no policy
I20260808 22:30:48.626499 4608 heartbeater.cc:347] Connected to a master server at 127.5.11.254:43289
I20260808 22:30:48.626724 4608 heartbeater.cc:464] Registering TS with master...
I20260808 22:30:48.627193 4608 heartbeater.cc:511] Master 127.5.11.254:43289 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.195@KRBTEST.COM" created.
I20260808 22:30:48.628343 4073 ts_manager.cc:194] Registered new tserver with Master: 64cd91397ca54ea59ef8324dd977cd80 (127.5.11.194:41845)
I20260808 22:30:48.628986 4073 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.194@KRBTEST.COM'} at 127.5.11.194:36531
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab.
Entry for principal kudu/127.5.11.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab.
I20260808 22:30:48.644367 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.195:0
--local_ip_for_outbound_sockets=127.5.11.195
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:43289
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:46391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:30:48.755688 4615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:30:48.755998 4615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:30:48.756079 4615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:30:48.760051 4615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:30:48.760126 4615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:30:48.760222 4615 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.195
I20260808 22:30:48.765265 4615 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:46391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.195.keytab
--principal=kudu/127.5.11.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:43289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.195
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:30:48.766260 4615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:30:48.767262 4615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:30:48.776700 4625 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:48.776702 4627 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:48.776749 4624 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:48.777065 4615 server_base.cc:1034] running on GCE node
I20260808 22:30:48.777436 4615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:30:48.778043 4615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:30:48.779208 4615 hybrid_clock.cc:648] HybridClock initialized: now 1786228248779188 us; error 29 us; skew 500 ppm
Aug 08 22:30:48 dist-test-slave-7f01 krb5kdc[3984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228248, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:30:48.781984 4615 init.cc:377] Logged in from keytab as kudu/127.5.11.195@KRBTEST.COM (short username kudu)
I20260808 22:30:48.782807 4615 webserver.cc:460] Webserver started at http://127.5.11.195:45979/ using document root <none> and password file <none>
I20260808 22:30:48.783352 4615 fs_manager.cc:359] Metadata directory not provided
I20260808 22:30:48.783455 4615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:30:48.783694 4615 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:30:48.785669 4615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-2/data/instance:
uuid: "83e51a0b0e0b4767898ab3c4a8b8cc47"
format_stamp: "Formatted at 2026-08-08 22:30:48 on dist-test-slave-7f01"
server_key: "e807c5811daea3bb7f96e9c451106166"
server_key_iv: "cf4fec5c4707f40b83e3c6331ec47760"
server_key_version: "encryptionkey@0"
I20260808 22:30:48.786232 4615 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance:
uuid: "83e51a0b0e0b4767898ab3c4a8b8cc47"
format_stamp: "Formatted at 2026-08-08 22:30:48 on dist-test-slave-7f01"
server_key: "e807c5811daea3bb7f96e9c451106166"
server_key_iv: "cf4fec5c4707f40b83e3c6331ec47760"
server_key_version: "encryptionkey@0"
I20260808 22:30:48.789949 4615 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 22:30:48.792330 4638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:48.793484 4615 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:30:48.793623 4615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-2/wal
uuid: "83e51a0b0e0b4767898ab3c4a8b8cc47"
format_stamp: "Formatted at 2026-08-08 22:30:48 on dist-test-slave-7f01"
server_key: "e807c5811daea3bb7f96e9c451106166"
server_key_iv: "cf4fec5c4707f40b83e3c6331ec47760"
server_key_version: "encryptionkey@0"
I20260808 22:30:48.793732 4615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:30:48.816351 4615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:30:48.819155 4615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:30:48.819350 4615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:30:48.820086 4615 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:30:48.821079 4615 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:30:48.821177 4615 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:48.821285 4615 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:30:48.821345 4615 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:48.830519 4615 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.195:38895
I20260808 22:30:48.830543 4816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.195:38895 every 8 connection(s)
I20260808 22:30:48.831498 4615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
Aug 08 22:30:48 dist-test-slave-7f01 krb5kdc[3984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228248, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.195@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:30:48.841219 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 4615
I20260808 22:30:48.841378 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance
I20260808 22:30:48.841678 5167 external_mini_cluster.cc:1428] Setting key c22defab3784899155bcc3ee7b3a4b4c
I20260808 22:30:48.845167 4133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:48.833341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:51717 (local address 127.5.11.254:43289)
0808 22:30:48.833476 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:48.833481 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:48.834292 (+ 811us) server_negotiation.cc:407] Connection header received
0808 22:30:48.835641 (+ 1349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:48.835645 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:48.835704 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:48.835799 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:48.836922 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:48.837468 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:48.838587 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:48.838815 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:48.838931 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:48.841358 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:48.841362 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:48.841365 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:30:48.841397 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:48.843156 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:48.843627 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:48.843630 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:48.843632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:48.843733 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:48.844190 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:48.844193 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:48.844194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:48.844371 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:48.844457 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:48.844876 (+ 419us) server_negotiation.cc:299] Negotiation successful
0808 22:30:48.844976 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260808 22:30:48.845882 4819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:48.833723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:43289 (local address 127.5.11.195:51717)
0808 22:30:48.834150 (+ 427us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:48.834191 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:48.835422 (+ 1231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:48.835969 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:48.835976 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:48.836311 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:30:48.836772 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:48.836785 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:48.837598 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:48.837601 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:48.838451 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:48.838460 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:48.838965 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:48.838969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:48.839140 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:48.839721 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:30:48.839787 (+ 66us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:30:48.841190 (+ 1403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:30:48.843258 (+ 2068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:30:48.843262 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:30:48.843273 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:30:48.843527 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:30:48.843864 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:30:48.843867 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:30:48.843869 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:30:48.844094 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:30:48.844460 (+ 366us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:30:48.844464 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:30:48.844674 (+ 210us) client_negotiation.cc:769] Sending connection context
0808 22:30:48.844823 (+ 149us) client_negotiation.cc:240] Negotiation successful
0808 22:30:48.845064 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":109,"threads_started":1}
I20260808 22:30:48.846891 4817 heartbeater.cc:347] Connected to a master server at 127.5.11.254:43289
I20260808 22:30:48.847086 4817 heartbeater.cc:464] Registering TS with master...
I20260808 22:30:48.847498 4817 heartbeater.cc:511] Master 127.5.11.254:43289 requested a full tablet report, sending...
I20260808 22:30:48.848475 4073 ts_manager.cc:194] Registered new tserver with Master: 83e51a0b0e0b4767898ab3c4a8b8cc47 (127.5.11.195:38895)
I20260808 22:30:48.849035 4073 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.5.11.195@KRBTEST.COM'} at 127.5.11.195:51717
I20260808 22:30:48.855647 5167 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:30:48.865113 4133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:48.857703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46842 (local address 127.5.11.254:43289)
0808 22:30:48.857832 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:48.857836 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:48.858001 (+ 165us) server_negotiation.cc:407] Connection header received
0808 22:30:48.858155 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:48.858160 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:48.858203 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:48.858283 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:48.859026 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:48.859631 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:48.860636 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:48.860838 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:48.860944 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:48.862223 (+ 1279us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:48.862226 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:48.862229 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:30:48.862260 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:48.863386 (+ 1126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:48.863850 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:48.863854 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:48.863856 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:48.863939 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:48.864227 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:48.864231 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:48.864233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:48.864457 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:48.864601 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:48.864840 (+ 239us) server_negotiation.cc:299] Negotiation successful
0808 22:30:48.864948 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 22:30:48.867689 4073 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46842:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:30:48.870756 4073 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:30:48.885087 4835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:48.879205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:38895 (local address 127.0.0.1:59210)
0808 22:30:48.879593 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:48.879636 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:48.879804 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:48.880744 (+ 940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:48.880751 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:48.880778 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:30:48.880917 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:48.880926 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:48.882109 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:48.882113 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:48.883518 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:48.883529 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:48.884522 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:48.884526 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:48.884638 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:48.884686 (+ 48us) client_negotiation.cc:769] Sending connection context
0808 22:30:48.884775 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 22:30:48.884902 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":65,"thread_start_us":76,"threads_started":1}
I20260808 22:30:48.885479 4837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:48.879938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:41603 (local address 127.0.0.1:41254)
0808 22:30:48.880281 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:48.880294 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:48.880392 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:48.881550 (+ 1158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:48.881554 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:48.881573 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:30:48.881663 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:48.881669 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:48.882727 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:48.882733 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:48.883937 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:48.883947 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:48.885060 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:48.885063 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:48.885151 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:48.885177 (+ 26us) client_negotiation.cc:769] Sending connection context
0808 22:30:48.885256 (+ 79us) client_negotiation.cc:240] Negotiation successful
0808 22:30:48.885323 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":1,"thread_start_us":112,"threads_started":1}
I20260808 22:30:48.885553 4840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:48.880700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41254 (local address 127.5.11.193:41603)
0808 22:30:48.881077 (+ 377us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:48.881095 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:48.881120 (+ 25us) server_negotiation.cc:407] Connection header received
0808 22:30:48.881199 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:48.881204 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:48.881385 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:48.881538 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:30:48.881785 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:48.882580 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:48.884073 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:48.884925 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:48.885035 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:48.885188 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:48.885281 (+ 93us) server_negotiation.cc:299] Negotiation successful
0808 22:30:48.885372 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":89,"threads_started":1}
I20260808 22:30:48.886049 4838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:48.879956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:41845 (local address 127.0.0.1:53980)
0808 22:30:48.880934 (+ 978us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:48.880948 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:48.881049 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:48.881740 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:48.881745 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:48.881768 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:30:48.881859 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:48.881866 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:48.883643 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:48.883645 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:48.884501 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:48.884510 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:48.885629 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:48.885634 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:48.885733 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:48.885756 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 22:30:48.885858 (+ 102us) client_negotiation.cc:240] Negotiation successful
0808 22:30:48.885925 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":915,"thread_start_us":119,"threads_started":1}
I20260808 22:30:48.886153 4839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:48.880273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53980 (local address 127.5.11.194:41845)
0808 22:30:48.880791 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:48.880805 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:48.881143 (+ 338us) server_negotiation.cc:407] Connection header received
0808 22:30:48.881214 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:48.881219 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:48.881385 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:48.881538 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:30:48.881994 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:48.883301 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:48.884621 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:48.885318 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:48.885414 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:48.885554 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:48.885895 (+ 341us) server_negotiation.cc:299] Negotiation successful
0808 22:30:48.885995 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":146,"threads_started":1}
I20260808 22:30:48.886466 4836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:48.879356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59210 (local address 127.5.11.195:38895)
0808 22:30:48.880237 (+ 881us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:48.880250 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:48.880274 (+ 24us) server_negotiation.cc:407] Connection header received
0808 22:30:48.880350 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:48.880355 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:48.880583 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:48.880754 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:30:48.881033 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:48.881956 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:48.883631 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:48.884377 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:48.885975 (+ 1598us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:48.886114 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:48.886161 (+ 47us) server_negotiation.cc:299] Negotiation successful
0808 22:30:48.886242 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":779,"thread_start_us":204,"threads_started":1}
I20260808 22:30:48.888729 4265 tablet_service.cc:1467] Processing CreateTablet for tablet 30ef19fcf3484abb8c051c38613720db (DEFAULT_TABLE table=test-table [id=963672f4eb6e4ae78b465b7939e6a117]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:30:48.889714 4265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30ef19fcf3484abb8c051c38613720db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:30:48.889415 4711 tablet_service.cc:1467] Processing CreateTablet for tablet 30ef19fcf3484abb8c051c38613720db (DEFAULT_TABLE table=test-table [id=963672f4eb6e4ae78b465b7939e6a117]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:30:48.889411 4491 tablet_service.cc:1467] Processing CreateTablet for tablet 30ef19fcf3484abb8c051c38613720db (DEFAULT_TABLE table=test-table [id=963672f4eb6e4ae78b465b7939e6a117]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:30:48.890439 4491 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30ef19fcf3484abb8c051c38613720db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:30:48.890467 4711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 30ef19fcf3484abb8c051c38613720db. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:30:48.895861 4842 tablet_bootstrap.cc:492] T 30ef19fcf3484abb8c051c38613720db P 36345df1409c4f50bc02e3bdfa020b5d: Bootstrap starting.
I20260808 22:30:48.897810 4846 tablet_bootstrap.cc:492] T 30ef19fcf3484abb8c051c38613720db P 83e51a0b0e0b4767898ab3c4a8b8cc47: Bootstrap starting.
I20260808 22:30:48.898846 4842 tablet_bootstrap.cc:654] T 30ef19fcf3484abb8c051c38613720db P 36345df1409c4f50bc02e3bdfa020b5d: Neither blocks nor log segments found. Creating new log.
I20260808 22:30:48.899611 4842 log.cc:824] T 30ef19fcf3484abb8c051c38613720db P 36345df1409c4f50bc02e3bdfa020b5d: Log is configured to *not* fsync() on all Append() calls
I20260808 22:30:48.899643 4846 tablet_bootstrap.cc:654] T 30ef19fcf3484abb8c051c38613720db P 83e51a0b0e0b4767898ab3c4a8b8cc47: Neither blocks nor log segments found. Creating new log.
I20260808 22:30:48.900686 4846 log.cc:824] T 30ef19fcf3484abb8c051c38613720db P 83e51a0b0e0b4767898ab3c4a8b8cc47: Log is configured to *not* fsync() on all Append() calls
I20260808 22:30:48.901372 4850 tablet_bootstrap.cc:492] T 30ef19fcf3484abb8c051c38613720db P 64cd91397ca54ea59ef8324dd977cd80: Bootstrap starting.
I20260808 22:30:48.901419 4842 tablet_bootstrap.cc:492] T 30ef19fcf3484abb8c051c38613720db P 36345df1409c4f50bc02e3bdfa020b5d: No bootstrap required, opened a new log
I20260808 22:30:48.901608 4842 ts_tablet_manager.cc:1397] T 30ef19fcf3484abb8c051c38613720db P 36345df1409c4f50bc02e3bdfa020b5d: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260808 22:30:48.902697 4846 tablet_bootstrap.cc:492] T 30ef19fcf3484abb8c051c38613720db P 83e51a0b0e0b4767898ab3c4a8b8cc47: No bootstrap required, opened a new log
I20260808 22:30:48.902961 4846 ts_tablet_manager.cc:1397] T 30ef19fcf3484abb8c051c38613720db P 83e51a0b0e0b4767898ab3c4a8b8cc47: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260808 22:30:48.903334 4850 tablet_bootstrap.cc:654] T 30ef19fcf3484abb8c051c38613720db P 64cd91397ca54ea59ef8324dd977cd80: Neither blocks nor log segments found. Creating new log.
I20260808 22:30:48.904363 4850 log.cc:824] T 30ef19fcf3484abb8c051c38613720db P 64cd91397ca54ea59ef8324dd977cd80: Log is configured to *not* fsync() on all Append() calls
I20260808 22:30:48.905867 4850 tablet_bootstrap.cc:492] T 30ef19fcf3484abb8c051c38613720db P 64cd91397ca54ea59ef8324dd977cd80: No bootstrap required, opened a new log
I20260808 22:30:48.906064 4850 ts_tablet_manager.cc:1397] T 30ef19fcf3484abb8c051c38613720db P 64cd91397ca54ea59ef8324dd977cd80: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 22:30:48.906080 4842 raft_consensus.cc:348] T 30ef19fcf3484abb8c051c38613720db P 36345df1409c4f50bc02e3bdfa020b5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83e51a0b0e0b4767898ab3c4a8b8cc47" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38895 } } peers { permanent_uuid: "36345df1409c4f50bc02e3bdfa020b5d" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 41603 } } peers { permanent_uuid: "64cd91397ca54ea59ef8324dd977cd80" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 41845 } }
I20260808 22:30:48.906349 4842 raft_consensus.cc:374] T 30ef19fcf3484abb8c051c38613720db P 36345df1409c4f50bc02e3bdfa020b5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:30:48.906456 4842 raft_consensus.cc:729] T 30ef19fcf3484abb8c051c38613720db P 36345df1409c4f50bc02e3bdfa020b5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 36345df1409c4f50bc02e3bdfa020b5d, State: Initialized, Role: FOLLOWER
I20260808 22:30:48.906807 4846 raft_consensus.cc:348] T 30ef19fcf3484abb8c051c38613720db P 83e51a0b0e0b4767898ab3c4a8b8cc47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83e51a0b0e0b4767898ab3c4a8b8cc47" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38895 } } peers { permanent_uuid: "36345df1409c4f50bc02e3bdfa020b5d" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 41603 } } peers { permanent_uuid: "64cd91397ca54ea59ef8324dd977cd80" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 41845 } }
I20260808 22:30:48.906957 4842 consensus_queue.cc:260] T 30ef19fcf3484abb8c051c38613720db P 36345df1409c4f50bc02e3bdfa020b5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83e51a0b0e0b4767898ab3c4a8b8cc47" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38895 } } peers { permanent_uuid: "36345df1409c4f50bc02e3bdfa020b5d" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 41603 } } peers { permanent_uuid: "64cd91397ca54ea59ef8324dd977cd80" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 41845 } }
I20260808 22:30:48.907073 4846 raft_consensus.cc:374] T 30ef19fcf3484abb8c051c38613720db P 83e51a0b0e0b4767898ab3c4a8b8cc47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:30:48.907155 4846 raft_consensus.cc:729] T 30ef19fcf3484abb8c051c38613720db P 83e51a0b0e0b4767898ab3c4a8b8cc47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83e51a0b0e0b4767898ab3c4a8b8cc47, State: Initialized, Role: FOLLOWER
I20260808 22:30:48.907615 4846 consensus_queue.cc:260] T 30ef19fcf3484abb8c051c38613720db P 83e51a0b0e0b4767898ab3c4a8b8cc47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83e51a0b0e0b4767898ab3c4a8b8cc47" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38895 } } peers { permanent_uuid: "36345df1409c4f50bc02e3bdfa020b5d" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 41603 } } peers { permanent_uuid: "64cd91397ca54ea59ef8324dd977cd80" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 41845 } }
I20260808 22:30:48.907996 4842 ts_tablet_manager.cc:1428] T 30ef19fcf3484abb8c051c38613720db P 36345df1409c4f50bc02e3bdfa020b5d: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260808 22:30:48.908286 4817 heartbeater.cc:503] Master 127.5.11.254:43289 was elected leader, sending a full tablet report...
I20260808 22:30:48.908366 4375 heartbeater.cc:503] Master 127.5.11.254:43289 was elected leader, sending a full tablet report...
I20260808 22:30:48.908560 4846 ts_tablet_manager.cc:1428] T 30ef19fcf3484abb8c051c38613720db P 83e51a0b0e0b4767898ab3c4a8b8cc47: Time spent starting tablet: real 0.006s user 0.002s sys 0.003s
I20260808 22:30:48.909691 4850 raft_consensus.cc:348] T 30ef19fcf3484abb8c051c38613720db P 64cd91397ca54ea59ef8324dd977cd80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83e51a0b0e0b4767898ab3c4a8b8cc47" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38895 } } peers { permanent_uuid: "36345df1409c4f50bc02e3bdfa020b5d" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 41603 } } peers { permanent_uuid: "64cd91397ca54ea59ef8324dd977cd80" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 41845 } }
I20260808 22:30:48.909947 4850 raft_consensus.cc:374] T 30ef19fcf3484abb8c051c38613720db P 64cd91397ca54ea59ef8324dd977cd80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:30:48.910028 4850 raft_consensus.cc:729] T 30ef19fcf3484abb8c051c38613720db P 64cd91397ca54ea59ef8324dd977cd80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64cd91397ca54ea59ef8324dd977cd80, State: Initialized, Role: FOLLOWER
I20260808 22:30:48.910454 4850 consensus_queue.cc:260] T 30ef19fcf3484abb8c051c38613720db P 64cd91397ca54ea59ef8324dd977cd80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83e51a0b0e0b4767898ab3c4a8b8cc47" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38895 } } peers { permanent_uuid: "36345df1409c4f50bc02e3bdfa020b5d" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 41603 } } peers { permanent_uuid: "64cd91397ca54ea59ef8324dd977cd80" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 41845 } }
I20260808 22:30:48.911095 4608 heartbeater.cc:503] Master 127.5.11.254:43289 was elected leader, sending a full tablet report...
I20260808 22:30:48.911414 4850 ts_tablet_manager.cc:1428] T 30ef19fcf3484abb8c051c38613720db P 64cd91397ca54ea59ef8324dd977cd80: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 22:30:48.981096 4857 raft_consensus.cc:482] T 30ef19fcf3484abb8c051c38613720db P 83e51a0b0e0b4767898ab3c4a8b8cc47 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:30:48.981326 4857 raft_consensus.cc:504] T 30ef19fcf3484abb8c051c38613720db P 83e51a0b0e0b4767898ab3c4a8b8cc47 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83e51a0b0e0b4767898ab3c4a8b8cc47" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38895 } } peers { permanent_uuid: "36345df1409c4f50bc02e3bdfa020b5d" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 41603 } } peers { permanent_uuid: "64cd91397ca54ea59ef8324dd977cd80" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 41845 } }
I20260808 22:30:48.982750 4857 leader_election.cc:288] T 30ef19fcf3484abb8c051c38613720db P 83e51a0b0e0b4767898ab3c4a8b8cc47 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 36345df1409c4f50bc02e3bdfa020b5d (127.5.11.193:41603), 64cd91397ca54ea59ef8324dd977cd80 (127.5.11.194:41845)
I20260808 22:30:48.986490 4819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:48.982933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:41603 (local address 127.5.11.195:49357)
0808 22:30:48.983215 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:48.983231 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:48.983328 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:48.983605 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:48.983608 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:48.983628 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:30:48.983782 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:48.983792 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:48.984522 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:48.984528 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:48.985366 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:48.985376 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:48.986048 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:48.986052 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:48.986158 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:48.986180 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 22:30:48.986281 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 22:30:48.986353 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":19}
I20260808 22:30:48.986490 4840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:48.983154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:49357 (local address 127.5.11.193:41603)
0808 22:30:48.983279 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:48.983284 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:48.983299 (+ 15us) server_negotiation.cc:407] Connection header received
0808 22:30:48.983438 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:48.983452 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:48.983497 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:48.983588 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:30:48.983891 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:48.984395 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:48.985515 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:48.985925 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:48.986058 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:48.986119 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:48.986301 (+ 182us) server_negotiation.cc:299] Negotiation successful
0808 22:30:48.986376 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260808 22:30:48.987079 4860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:48.983548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:41845 (local address 127.5.11.195:50561)
0808 22:30:48.983895 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:48.983907 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:48.984034 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:48.984275 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:48.984278 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:48.984294 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:30:48.984414 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:48.984419 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:48.985114 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:48.985118 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:48.985923 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:48.985932 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:48.986604 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:48.986608 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:48.986704 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:48.986724 (+ 20us) client_negotiation.cc:769] Sending connection context
0808 22:30:48.986863 (+ 139us) client_negotiation.cc:240] Negotiation successful
0808 22:30:48.986929 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":21,"thread_start_us":141,"threads_started":1}
I20260808 22:30:48.987126 4839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:48.983292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:50561 (local address 127.5.11.194:41845)
0808 22:30:48.983440 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:48.983443 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:48.983998 (+ 555us) server_negotiation.cc:407] Connection header received
0808 22:30:48.984122 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:48.984125 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:48.984174 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:48.984254 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:30:48.984524 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:48.985003 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:48.986054 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:48.986462 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:48.986604 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:48.986650 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:48.986898 (+ 248us) server_negotiation.cc:299] Negotiation successful
0808 22:30:48.987008 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 22:30:48.987272 4301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30ef19fcf3484abb8c051c38613720db" candidate_uuid: "83e51a0b0e0b4767898ab3c4a8b8cc47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36345df1409c4f50bc02e3bdfa020b5d" is_pre_election: true
I20260808 22:30:48.987617 4301 raft_consensus.cc:2454] T 30ef19fcf3484abb8c051c38613720db P 36345df1409c4f50bc02e3bdfa020b5d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83e51a0b0e0b4767898ab3c4a8b8cc47 in term 0.
I20260808 22:30:48.988000 4525 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30ef19fcf3484abb8c051c38613720db" candidate_uuid: "83e51a0b0e0b4767898ab3c4a8b8cc47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64cd91397ca54ea59ef8324dd977cd80" is_pre_election: true
I20260808 22:30:48.988152 4641 leader_election.cc:302] T 30ef19fcf3484abb8c051c38613720db P 83e51a0b0e0b4767898ab3c4a8b8cc47 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36345df1409c4f50bc02e3bdfa020b5d, 83e51a0b0e0b4767898ab3c4a8b8cc47; no voters:
I20260808 22:30:48.988224 4525 raft_consensus.cc:2454] T 30ef19fcf3484abb8c051c38613720db P 64cd91397ca54ea59ef8324dd977cd80 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83e51a0b0e0b4767898ab3c4a8b8cc47 in term 0.
I20260808 22:30:48.988456 4857 raft_consensus.cc:2781] T 30ef19fcf3484abb8c051c38613720db P 83e51a0b0e0b4767898ab3c4a8b8cc47 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:30:48.988566 4857 raft_consensus.cc:482] T 30ef19fcf3484abb8c051c38613720db P 83e51a0b0e0b4767898ab3c4a8b8cc47 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:30:48.988618 4857 raft_consensus.cc:3037] T 30ef19fcf3484abb8c051c38613720db P 83e51a0b0e0b4767898ab3c4a8b8cc47 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:30:48.989650 4857 raft_consensus.cc:504] T 30ef19fcf3484abb8c051c38613720db P 83e51a0b0e0b4767898ab3c4a8b8cc47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83e51a0b0e0b4767898ab3c4a8b8cc47" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38895 } } peers { permanent_uuid: "36345df1409c4f50bc02e3bdfa020b5d" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 41603 } } peers { permanent_uuid: "64cd91397ca54ea59ef8324dd977cd80" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 41845 } }
I20260808 22:30:48.989975 4857 leader_election.cc:288] T 30ef19fcf3484abb8c051c38613720db P 83e51a0b0e0b4767898ab3c4a8b8cc47 [CANDIDATE]: Term 1 election: Requested vote from peers 36345df1409c4f50bc02e3bdfa020b5d (127.5.11.193:41603), 64cd91397ca54ea59ef8324dd977cd80 (127.5.11.194:41845)
I20260808 22:30:48.990288 4301 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30ef19fcf3484abb8c051c38613720db" candidate_uuid: "83e51a0b0e0b4767898ab3c4a8b8cc47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "36345df1409c4f50bc02e3bdfa020b5d"
I20260808 22:30:48.990340 4525 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "30ef19fcf3484abb8c051c38613720db" candidate_uuid: "83e51a0b0e0b4767898ab3c4a8b8cc47" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "64cd91397ca54ea59ef8324dd977cd80"
I20260808 22:30:48.990428 4301 raft_consensus.cc:3037] T 30ef19fcf3484abb8c051c38613720db P 36345df1409c4f50bc02e3bdfa020b5d [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:30:48.990476 4525 raft_consensus.cc:3037] T 30ef19fcf3484abb8c051c38613720db P 64cd91397ca54ea59ef8324dd977cd80 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:30:48.991439 4301 raft_consensus.cc:2454] T 30ef19fcf3484abb8c051c38613720db P 36345df1409c4f50bc02e3bdfa020b5d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83e51a0b0e0b4767898ab3c4a8b8cc47 in term 1.
I20260808 22:30:48.991580 4525 raft_consensus.cc:2454] T 30ef19fcf3484abb8c051c38613720db P 64cd91397ca54ea59ef8324dd977cd80 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83e51a0b0e0b4767898ab3c4a8b8cc47 in term 1.
I20260808 22:30:48.991859 4641 leader_election.cc:302] T 30ef19fcf3484abb8c051c38613720db P 83e51a0b0e0b4767898ab3c4a8b8cc47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 36345df1409c4f50bc02e3bdfa020b5d, 83e51a0b0e0b4767898ab3c4a8b8cc47; no voters:
I20260808 22:30:48.992128 4857 raft_consensus.cc:2781] T 30ef19fcf3484abb8c051c38613720db P 83e51a0b0e0b4767898ab3c4a8b8cc47 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:30:48.992375 4857 raft_consensus.cc:686] T 30ef19fcf3484abb8c051c38613720db P 83e51a0b0e0b4767898ab3c4a8b8cc47 [term 1 LEADER]: Becoming Leader. State: Replica: 83e51a0b0e0b4767898ab3c4a8b8cc47, State: Running, Role: LEADER
I20260808 22:30:48.992705 4857 consensus_queue.cc:237] T 30ef19fcf3484abb8c051c38613720db P 83e51a0b0e0b4767898ab3c4a8b8cc47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83e51a0b0e0b4767898ab3c4a8b8cc47" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38895 } } peers { permanent_uuid: "36345df1409c4f50bc02e3bdfa020b5d" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 41603 } } peers { permanent_uuid: "64cd91397ca54ea59ef8324dd977cd80" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 41845 } }
I20260808 22:30:48.995702 4065 catalog_manager.cc:5495] T 30ef19fcf3484abb8c051c38613720db P 83e51a0b0e0b4767898ab3c4a8b8cc47 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83e51a0b0e0b4767898ab3c4a8b8cc47 (127.5.11.195). New cstate: current_term: 1 leader_uuid: "83e51a0b0e0b4767898ab3c4a8b8cc47" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83e51a0b0e0b4767898ab3c4a8b8cc47" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 38895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "36345df1409c4f50bc02e3bdfa020b5d" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 41603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "64cd91397ca54ea59ef8324dd977cd80" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 41845 } health_report { overall_health: UNKNOWN } } }
I20260808 22:30:49.007046 4836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:49.003222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59220 (local address 127.5.11.195:38895)
0808 22:30:49.003347 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:49.003353 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:49.003407 (+ 54us) server_negotiation.cc:407] Connection header received
0808 22:30:49.003588 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:49.003591 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:49.003642 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:49.003745 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:30:49.004027 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:49.004518 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:49.005445 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:49.005668 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:49.005771 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:49.006167 (+ 396us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:30:49.006632 (+ 465us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:30:49.006757 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:49.006840 (+ 83us) server_negotiation.cc:299] Negotiation successful
0808 22:30:49.006895 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260808 22:30:49.013016 4301 raft_consensus.cc:1261] T 30ef19fcf3484abb8c051c38613720db P 36345df1409c4f50bc02e3bdfa020b5d [term 1 FOLLOWER]: Refusing update from remote peer 83e51a0b0e0b4767898ab3c4a8b8cc47: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:30:49.013016 4525 raft_consensus.cc:1261] T 30ef19fcf3484abb8c051c38613720db P 64cd91397ca54ea59ef8324dd977cd80 [term 1 FOLLOWER]: Refusing update from remote peer 83e51a0b0e0b4767898ab3c4a8b8cc47: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:30:49.013708 4863 consensus_queue.cc:1035] T 30ef19fcf3484abb8c051c38613720db P 83e51a0b0e0b4767898ab3c4a8b8cc47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64cd91397ca54ea59ef8324dd977cd80" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 41845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:30:49.013886 4857 consensus_queue.cc:1035] T 30ef19fcf3484abb8c051c38613720db P 83e51a0b0e0b4767898ab3c4a8b8cc47 [LEADER]: Connected to new peer: Peer: permanent_uuid: "36345df1409c4f50bc02e3bdfa020b5d" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 41603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:30:49.029181 4870 mvcc.cc:205] Tried to move back new op lower bound from 7316390907950014464 to 7316390907876515840. Current Snapshot: MvccSnapshot[applied={T|T < 7316390907950014464}]
I20260808 22:30:49.030149 4868 mvcc.cc:205] Tried to move back new op lower bound from 7316390907950014464 to 7316390907876515840. Current Snapshot: MvccSnapshot[applied={T|T < 7316390907950014464 or (T in {7316390907950014464})}]
I20260808 22:30:49.036069 4073 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:30:49.036208 4073 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46842:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:30:49.038401 4073 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8c15714816594854a5f22bb2e6e2e4f2: Sending DeleteTablet for 3 replicas of tablet 30ef19fcf3484abb8c051c38613720db
I20260808 22:30:49.039366 4711 tablet_service.cc:1514] Processing DeleteTablet for tablet 30ef19fcf3484abb8c051c38613720db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:30:49 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:59210
I20260808 22:30:49.039373 4491 tablet_service.cc:1514] Processing DeleteTablet for tablet 30ef19fcf3484abb8c051c38613720db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:30:49 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:53980
I20260808 22:30:49.039870 4882 tablet_replica.cc:330] T 30ef19fcf3484abb8c051c38613720db P 83e51a0b0e0b4767898ab3c4a8b8cc47: stopping tablet replica
I20260808 22:30:49.039875 4884 tablet_replica.cc:330] T 30ef19fcf3484abb8c051c38613720db P 64cd91397ca54ea59ef8324dd977cd80: stopping tablet replica
I20260808 22:30:49.040200 4884 raft_consensus.cc:2229] T 30ef19fcf3484abb8c051c38613720db P 64cd91397ca54ea59ef8324dd977cd80 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:30:49.040203 4882 raft_consensus.cc:2229] T 30ef19fcf3484abb8c051c38613720db P 83e51a0b0e0b4767898ab3c4a8b8cc47 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:30:49.040299 4265 tablet_service.cc:1514] Processing DeleteTablet for tablet 30ef19fcf3484abb8c051c38613720db with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:30:49 UTC) from {username='kudu', principal='kudu/127.5.11.254@KRBTEST.COM'} at 127.0.0.1:41254
I20260808 22:30:49.040521 4882 raft_consensus.cc:2258] T 30ef19fcf3484abb8c051c38613720db P 83e51a0b0e0b4767898ab3c4a8b8cc47 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:30:49.040575 4884 raft_consensus.cc:2258] T 30ef19fcf3484abb8c051c38613720db P 64cd91397ca54ea59ef8324dd977cd80 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:30:49.040815 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 4169
I20260808 22:30:49.042673 4884 ts_tablet_manager.cc:1905] T 30ef19fcf3484abb8c051c38613720db P 64cd91397ca54ea59ef8324dd977cd80: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:30:49.045020 4882 ts_tablet_manager.cc:1905] T 30ef19fcf3484abb8c051c38613720db P 83e51a0b0e0b4767898ab3c4a8b8cc47: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:30:49.045595 4884 ts_tablet_manager.cc:1918] T 30ef19fcf3484abb8c051c38613720db P 64cd91397ca54ea59ef8324dd977cd80: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:30:49.045717 4884 log.cc:1196] T 30ef19fcf3484abb8c051c38613720db P 64cd91397ca54ea59ef8324dd977cd80: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-1/wal/wals/30ef19fcf3484abb8c051c38613720db
I20260808 22:30:49.046130 4884 ts_tablet_manager.cc:1939] T 30ef19fcf3484abb8c051c38613720db P 64cd91397ca54ea59ef8324dd977cd80: Deleting consensus metadata
I20260808 22:30:49.047379 4036 catalog_manager.cc:4841] TS 64cd91397ca54ea59ef8324dd977cd80 (127.5.11.194:41845): tablet 30ef19fcf3484abb8c051c38613720db (table test-table [id=963672f4eb6e4ae78b465b7939e6a117]) successfully deleted
I20260808 22:30:49.048547 4882 ts_tablet_manager.cc:1918] T 30ef19fcf3484abb8c051c38613720db P 83e51a0b0e0b4767898ab3c4a8b8cc47: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:30:49.048676 4882 log.cc:1196] T 30ef19fcf3484abb8c051c38613720db P 83e51a0b0e0b4767898ab3c4a8b8cc47: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786227991988852-5167-0/minicluster-data/ts-2/wal/wals/30ef19fcf3484abb8c051c38613720db
I20260808 22:30:49.049094 4882 ts_tablet_manager.cc:1939] T 30ef19fcf3484abb8c051c38613720db P 83e51a0b0e0b4767898ab3c4a8b8cc47: Deleting consensus metadata
I20260808 22:30:49.050457 4039 catalog_manager.cc:4841] TS 83e51a0b0e0b4767898ab3c4a8b8cc47 (127.5.11.195:38895): tablet 30ef19fcf3484abb8c051c38613720db (table test-table [id=963672f4eb6e4ae78b465b7939e6a117]) successfully deleted
W20260808 22:30:49.060534 4039 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.11.193:41603 (error 108)
I20260808 22:30:49.060696 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 4380
I20260808 22:30:49.071272 4838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:49.070935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:41603 (local address 127.0.0.1:41264)
0808 22:30:49.071093 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:49.071171 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.193:41603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260808 22:30:49.071576 4039 catalog_manager.cc:4584] TS 36345df1409c4f50bc02e3bdfa020b5d (127.5.11.193:41603): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 30ef19fcf3484abb8c051c38613720db: Network error: Client connection negotiation failed: client connection to 127.5.11.193:41603: connect: Connection refused (error 111)
I20260808 22:30:49.076046 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 4615
I20260808 22:30:49.086410 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 4009
2026-08-08T22:30:49Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2554 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-08T22:30:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:30:49Z Disabled control of system clock
I20260808 22:30:49.122345 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:35993
--webserver_interface=127.5.11.254
--webserver_port=0
--builtin_ntp_servers=127.5.11.212:39859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:35993
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 22:30:49.233228 4892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:30:49.233462 4892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:30:49.233513 4892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:30:49.237406 4892 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:30:49.237453 4892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:30:49.237468 4892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:30:49.237485 4892 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:30:49.237500 4892 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:30:49.242044 4892 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:39859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:35993
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:35993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:30:49.242969 4892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:30:49.244222 4892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:30:49.252396 4900 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:49.252395 4899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:49.252436 4903 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:49.252630 4892 server_base.cc:1034] running on GCE node
I20260808 22:30:49.253304 4892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:30:49.254292 4892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:30:49.255455 4892 hybrid_clock.cc:648] HybridClock initialized: now 1786228249255428 us; error 42 us; skew 500 ppm
I20260808 22:30:49.257383 4892 webserver.cc:460] Webserver started at http://127.5.11.254:38331/ using document root <none> and password file <none>
I20260808 22:30:49.257927 4892 fs_manager.cc:359] Metadata directory not provided
I20260808 22:30:49.257995 4892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:30:49.258158 4892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:30:49.260156 4892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "b26dc241dcd34c60b584eb64ede93f8a"
format_stamp: "Formatted at 2026-08-08 22:30:49 on dist-test-slave-7f01"
server_key: "454f33dc612c7763c686440785b1665a"
server_key_iv: "9cedf25eb5bb4ae602610aa436315fee"
server_key_version: "encryptionkey@0"
I20260808 22:30:49.260661 4892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "b26dc241dcd34c60b584eb64ede93f8a"
format_stamp: "Formatted at 2026-08-08 22:30:49 on dist-test-slave-7f01"
server_key: "454f33dc612c7763c686440785b1665a"
server_key_iv: "9cedf25eb5bb4ae602610aa436315fee"
server_key_version: "encryptionkey@0"
I20260808 22:30:49.264128 4892 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:30:49.266376 4915 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:49.267618 4892 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:30:49.267719 4892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "b26dc241dcd34c60b584eb64ede93f8a"
format_stamp: "Formatted at 2026-08-08 22:30:49 on dist-test-slave-7f01"
server_key: "454f33dc612c7763c686440785b1665a"
server_key_iv: "9cedf25eb5bb4ae602610aa436315fee"
server_key_version: "encryptionkey@0"
I20260808 22:30:49.267865 4892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:30:49.277454 4892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:30:49.278105 4892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:30:49.278251 4892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:30:49.285280 4892 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:35993
I20260808 22:30:49.285283 5012 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:35993 every 8 connection(s)
I20260808 22:30:49.286221 4892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:30:49.287964 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 4892
I20260808 22:30:49.288193 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:30:49.288516 5167 external_mini_cluster.cc:1428] Setting key 6f6519f64b065d49ecac6e2daf9b4c70
I20260808 22:30:49.289690 5014 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:30:49.296010 5014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b26dc241dcd34c60b584eb64ede93f8a: Bootstrap starting.
I20260808 22:30:49.297674 5018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:49.289950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38904 (local address 127.5.11.254:35993)
0808 22:30:49.290559 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:49.290575 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:49.290611 (+ 36us) server_negotiation.cc:407] Connection header received
0808 22:30:49.291256 (+ 645us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:49.291272 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:49.291645 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:49.291997 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:49.292331 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:49.293295 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:49.294437 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:49.294894 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:49.295022 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:49.295915 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:49.295920 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:49.295932 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:30:49.296368 (+ 436us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:49.296427 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:30:49.296448 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:49.296525 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:49.296697 (+ 172us) server_negotiation.cc:299] Negotiation successful
0808 22:30:49.297002 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":148,"threads_started":1}
I20260808 22:30:49.298354 5014 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b26dc241dcd34c60b584eb64ede93f8a: Neither blocks nor log segments found. Creating new log.
I20260808 22:30:49.299139 5014 log.cc:824] T 00000000000000000000000000000000 P b26dc241dcd34c60b584eb64ede93f8a: Log is configured to *not* fsync() on all Append() calls
I20260808 22:30:49.301095 5014 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b26dc241dcd34c60b584eb64ede93f8a: No bootstrap required, opened a new log
I20260808 22:30:49.304697 5014 raft_consensus.cc:348] T 00000000000000000000000000000000 P b26dc241dcd34c60b584eb64ede93f8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b26dc241dcd34c60b584eb64ede93f8a" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 35993 } }
I20260808 22:30:49.304912 5014 raft_consensus.cc:374] T 00000000000000000000000000000000 P b26dc241dcd34c60b584eb64ede93f8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:30:49.304955 5014 raft_consensus.cc:729] T 00000000000000000000000000000000 P b26dc241dcd34c60b584eb64ede93f8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b26dc241dcd34c60b584eb64ede93f8a, State: Initialized, Role: FOLLOWER
I20260808 22:30:49.305296 5014 consensus_queue.cc:260] T 00000000000000000000000000000000 P b26dc241dcd34c60b584eb64ede93f8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b26dc241dcd34c60b584eb64ede93f8a" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 35993 } }
I20260808 22:30:49.305405 5014 raft_consensus.cc:388] T 00000000000000000000000000000000 P b26dc241dcd34c60b584eb64ede93f8a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:30:49.305449 5014 raft_consensus.cc:482] T 00000000000000000000000000000000 P b26dc241dcd34c60b584eb64ede93f8a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:30:49.305517 5014 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b26dc241dcd34c60b584eb64ede93f8a [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:30:49.306300 5014 raft_consensus.cc:504] T 00000000000000000000000000000000 P b26dc241dcd34c60b584eb64ede93f8a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b26dc241dcd34c60b584eb64ede93f8a" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 35993 } }
I20260808 22:30:49.306578 5014 leader_election.cc:302] T 00000000000000000000000000000000 P b26dc241dcd34c60b584eb64ede93f8a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b26dc241dcd34c60b584eb64ede93f8a; no voters:
I20260808 22:30:49.306900 5014 leader_election.cc:288] T 00000000000000000000000000000000 P b26dc241dcd34c60b584eb64ede93f8a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:30:49.306994 5023 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b26dc241dcd34c60b584eb64ede93f8a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:30:49.307235 5023 raft_consensus.cc:686] T 00000000000000000000000000000000 P b26dc241dcd34c60b584eb64ede93f8a [term 1 LEADER]: Becoming Leader. State: Replica: b26dc241dcd34c60b584eb64ede93f8a, State: Running, Role: LEADER
I20260808 22:30:49.307536 5023 consensus_queue.cc:237] T 00000000000000000000000000000000 P b26dc241dcd34c60b584eb64ede93f8a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b26dc241dcd34c60b584eb64ede93f8a" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 35993 } }
I20260808 22:30:49.308265 5014 sys_catalog.cc:564] T 00000000000000000000000000000000 P b26dc241dcd34c60b584eb64ede93f8a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:30:49.309664 5026 sys_catalog.cc:455] T 00000000000000000000000000000000 P b26dc241dcd34c60b584eb64ede93f8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader b26dc241dcd34c60b584eb64ede93f8a. Latest consensus state: current_term: 1 leader_uuid: "b26dc241dcd34c60b584eb64ede93f8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b26dc241dcd34c60b584eb64ede93f8a" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 35993 } } }
I20260808 22:30:49.309633 5025 sys_catalog.cc:455] T 00000000000000000000000000000000 P b26dc241dcd34c60b584eb64ede93f8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b26dc241dcd34c60b584eb64ede93f8a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b26dc241dcd34c60b584eb64ede93f8a" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 35993 } } }
I20260808 22:30:49.309794 5026 sys_catalog.cc:458] T 00000000000000000000000000000000 P b26dc241dcd34c60b584eb64ede93f8a [sys.catalog]: This master's current role is: LEADER
I20260808 22:30:49.309829 5025 sys_catalog.cc:458] T 00000000000000000000000000000000 P b26dc241dcd34c60b584eb64ede93f8a [sys.catalog]: This master's current role is: LEADER
I20260808 22:30:49.310575 5042 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:30:49.313179 5042 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:30:49.318161 5042 catalog_manager.cc:1347] Generated new cluster ID: 8ebf3611138c422c9d8bf4012bf91d8c
I20260808 22:30:49.318228 5042 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:30:49.334501 5042 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:30:49.335362 5042 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:30:49.346432 5042 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b26dc241dcd34c60b584eb64ede93f8a: Generated new TSK 0
I20260808 22:30:49.347081 5042 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:30:49.350042 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.193:0
--local_ip_for_outbound_sockets=127.5.11.193
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:35993
--builtin_ntp_servers=127.5.11.212:39859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260808 22:30:49.463833 5058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:30:49.464073 5058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:30:49.464110 5058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:30:49.468058 5058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:30:49.468104 5058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:30:49.468161 5058 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.193
I20260808 22:30:49.472867 5058 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:39859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.5.11.193
--webserver_port=0
--tserver_master_addrs=127.5.11.254:35993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.193
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:30:49.473794 5058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:30:49.475077 5058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:30:49.484231 5075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:49.484223 5072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:49.484225 5069 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:49.484448 5058 server_base.cc:1034] running on GCE node
I20260808 22:30:49.484866 5058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:30:49.485491 5058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:30:49.486652 5058 hybrid_clock.cc:648] HybridClock initialized: now 1786228249486634 us; error 27 us; skew 500 ppm
I20260808 22:30:49.488456 5058 webserver.cc:460] Webserver started at http://127.5.11.193:42697/ using document root <none> and password file <none>
I20260808 22:30:49.489046 5058 fs_manager.cc:359] Metadata directory not provided
I20260808 22:30:49.489109 5058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:30:49.489364 5058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:30:49.491400 5058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-0/data/instance:
uuid: "7072be4f945e4694bfc9566773b28c0e"
format_stamp: "Formatted at 2026-08-08 22:30:49 on dist-test-slave-7f01"
server_key: "3a69d7f7d038f75ce3861d51daa0db2f"
server_key_iv: "b44a9612b5d8033dfe4fa18a8a68b9cc"
server_key_version: "encryptionkey@0"
I20260808 22:30:49.491989 5058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance:
uuid: "7072be4f945e4694bfc9566773b28c0e"
format_stamp: "Formatted at 2026-08-08 22:30:49 on dist-test-slave-7f01"
server_key: "3a69d7f7d038f75ce3861d51daa0db2f"
server_key_iv: "b44a9612b5d8033dfe4fa18a8a68b9cc"
server_key_version: "encryptionkey@0"
I20260808 22:30:49.495533 5058 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 22:30:49.497912 5084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:49.499070 5058 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:30:49.499249 5058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-0/wal
uuid: "7072be4f945e4694bfc9566773b28c0e"
format_stamp: "Formatted at 2026-08-08 22:30:49 on dist-test-slave-7f01"
server_key: "3a69d7f7d038f75ce3861d51daa0db2f"
server_key_iv: "b44a9612b5d8033dfe4fa18a8a68b9cc"
server_key_version: "encryptionkey@0"
I20260808 22:30:49.499377 5058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:30:49.520711 5058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:30:49.521375 5058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:30:49.521582 5058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:30:49.522151 5058 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:30:49.523078 5058 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:30:49.523151 5058 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:49.523223 5058 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:30:49.523267 5058 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:49.532445 5058 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.193:33527
I20260808 22:30:49.532475 5286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.193:33527 every 8 connection(s)
I20260808 22:30:49.533308 5058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-0/data/info.pb
I20260808 22:30:49.537189 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 5058
I20260808 22:30:49.537330 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-0/wal/instance
I20260808 22:30:49.537626 5167 external_mini_cluster.cc:1428] Setting key 1043fdddfa12dd76c9ac377bf08af105
I20260808 22:30:49.539781 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.194:0
--local_ip_for_outbound_sockets=127.5.11.194
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:35993
--builtin_ntp_servers=127.5.11.212:39859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 22:30:49.546834 5018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:49.535146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.193:44749 (local address 127.5.11.254:35993)
0808 22:30:49.535314 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:49.535320 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:49.536402 (+ 1082us) server_negotiation.cc:407] Connection header received
0808 22:30:49.537884 (+ 1482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:49.537887 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:49.537949 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:49.538047 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:49.539230 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:49.540058 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:49.541169 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:49.541573 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:49.541689 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:49.546182 (+ 4493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:49.546189 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:49.546195 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:30:49.546214 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:49.546243 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:30:49.546255 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:49.546340 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:49.546501 (+ 161us) server_negotiation.cc:299] Negotiation successful
0808 22:30:49.546594 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 22:30:49.547523 5290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:49.535554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:35993 (local address 127.5.11.193:44749)
0808 22:30:49.536251 (+ 697us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:49.536293 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:49.537644 (+ 1351us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:49.538279 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:49.538286 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:49.538394 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:30:49.539074 (+ 680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:49.539096 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:49.540180 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:49.540183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:49.541043 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:49.541053 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:49.541807 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:49.541815 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:49.542106 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:49.545854 (+ 3748us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:30:49.545856 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:30:49.545908 (+ 52us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:30:49.545991 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:30:49.546355 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:30:49.546363 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:30:49.546368 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:30:49.546458 (+ 90us) client_negotiation.cc:240] Negotiation successful
0808 22:30:49.546667 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":168,"threads_started":1}
I20260808 22:30:49.548772 5287 heartbeater.cc:347] Connected to a master server at 127.5.11.254:35993
I20260808 22:30:49.549041 5287 heartbeater.cc:464] Registering TS with master...
I20260808 22:30:49.549716 5287 heartbeater.cc:511] Master 127.5.11.254:35993 requested a full tablet report, sending...
I20260808 22:30:49.551312 4941 ts_manager.cc:194] Registered new tserver with Master: 7072be4f945e4694bfc9566773b28c0e (127.5.11.193:33527)
I20260808 22:30:49.552484 4941 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.11.193:44749
W20260808 22:30:49.654472 5291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:30:49.654709 5291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:30:49.654747 5291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:30:49.658772 5291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:30:49.658821 5291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:30:49.658923 5291 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.194
I20260808 22:30:49.663512 5291 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:39859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.5.11.194
--webserver_port=0
--tserver_master_addrs=127.5.11.254:35993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.194
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:30:49.664510 5291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:30:49.665839 5291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 22:30:49.675482 5291 server_base.cc:1034] running on GCE node
W20260808 22:30:49.675424 5302 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:49.675398 5299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:49.675398 5305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:49.676059 5291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:30:49.676697 5291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:30:49.677881 5291 hybrid_clock.cc:648] HybridClock initialized: now 1786228249677869 us; error 31 us; skew 500 ppm
I20260808 22:30:49.679589 5291 webserver.cc:460] Webserver started at http://127.5.11.194:36533/ using document root <none> and password file <none>
I20260808 22:30:49.680161 5291 fs_manager.cc:359] Metadata directory not provided
I20260808 22:30:49.680223 5291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:30:49.680388 5291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:30:49.682400 5291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-1/data/instance:
uuid: "ba1b9107fe864100a1e8cb108ddc0f3c"
format_stamp: "Formatted at 2026-08-08 22:30:49 on dist-test-slave-7f01"
server_key: "d5f5941cd8fbd5b0777568032dcb4d90"
server_key_iv: "d3771e04b2745ca47e8fc9d28380dad5"
server_key_version: "encryptionkey@0"
I20260808 22:30:49.682981 5291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance:
uuid: "ba1b9107fe864100a1e8cb108ddc0f3c"
format_stamp: "Formatted at 2026-08-08 22:30:49 on dist-test-slave-7f01"
server_key: "d5f5941cd8fbd5b0777568032dcb4d90"
server_key_iv: "d3771e04b2745ca47e8fc9d28380dad5"
server_key_version: "encryptionkey@0"
I20260808 22:30:49.686546 5291 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 22:30:49.689087 5313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:49.690219 5291 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:30:49.690348 5291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-1/wal
uuid: "ba1b9107fe864100a1e8cb108ddc0f3c"
format_stamp: "Formatted at 2026-08-08 22:30:49 on dist-test-slave-7f01"
server_key: "d5f5941cd8fbd5b0777568032dcb4d90"
server_key_iv: "d3771e04b2745ca47e8fc9d28380dad5"
server_key_version: "encryptionkey@0"
I20260808 22:30:49.690457 5291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:30:49.710714 5291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:30:49.711341 5291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:30:49.711544 5291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:30:49.712148 5291 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:30:49.713100 5291 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:30:49.713148 5291 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:49.713218 5291 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:30:49.713259 5291 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:49.722425 5291 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.194:40071
I20260808 22:30:49.722465 5525 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.194:40071 every 8 connection(s)
I20260808 22:30:49.723342 5291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-1/data/info.pb
I20260808 22:30:49.727497 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 5291
I20260808 22:30:49.727629 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-1/wal/instance
I20260808 22:30:49.727982 5167 external_mini_cluster.cc:1428] Setting key ffdfbe36f2d1ff9a5d5f422907e167ba
I20260808 22:30:49.730039 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.5.11.195:0
--local_ip_for_outbound_sockets=127.5.11.195
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:35993
--builtin_ntp_servers=127.5.11.212:39859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260808 22:30:49.732887 5018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:49.725144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.194:55769 (local address 127.5.11.254:35993)
0808 22:30:49.725293 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:49.725296 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:49.726266 (+ 970us) server_negotiation.cc:407] Connection header received
0808 22:30:49.727691 (+ 1425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:49.727694 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:49.727784 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:49.727892 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:49.728952 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:49.729540 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:49.730782 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:49.731174 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:49.731302 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:49.732449 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:49.732452 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:49.732454 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:30:49.732466 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:49.732486 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:30:49.732494 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:49.732554 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:49.732730 (+ 176us) server_negotiation.cc:299] Negotiation successful
0808 22:30:49.732779 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 22:30:49.734012 5529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:49.725467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:35993 (local address 127.5.11.194:55769)
0808 22:30:49.726114 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:49.726156 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:49.727481 (+ 1325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:49.728101 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:49.728110 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:49.728216 (+ 106us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:30:49.728806 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:49.728820 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:49.729671 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:49.729674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:49.730658 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:49.730669 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:49.731320 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:49.731323 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:49.731456 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:49.732140 (+ 684us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:30:49.732144 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:30:49.732207 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:30:49.732294 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:30:49.732580 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:30:49.732589 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:30:49.732597 (+ 8us) client_negotiation.cc:769] Sending connection context
0808 22:30:49.732725 (+ 128us) client_negotiation.cc:240] Negotiation successful
0808 22:30:49.733011 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":110,"threads_started":1}
I20260808 22:30:49.735587 5526 heartbeater.cc:347] Connected to a master server at 127.5.11.254:35993
I20260808 22:30:49.735868 5526 heartbeater.cc:464] Registering TS with master...
I20260808 22:30:49.736370 5526 heartbeater.cc:511] Master 127.5.11.254:35993 requested a full tablet report, sending...
I20260808 22:30:49.737481 4941 ts_manager.cc:194] Registered new tserver with Master: ba1b9107fe864100a1e8cb108ddc0f3c (127.5.11.194:40071)
I20260808 22:30:49.738334 4941 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.11.194:55769
W20260808 22:30:49.844537 5531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:30:49.844777 5531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:30:49.844813 5531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:30:49.848975 5531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:30:49.849025 5531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:30:49.849076 5531 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.5.11.195
I20260808 22:30:49.853739 5531 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:39859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.5.11.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.5.11.195
--webserver_port=0
--tserver_master_addrs=127.5.11.254:35993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.5.11.195
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:30:49.854648 5531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:30:49.855993 5531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:30:49.865221 5539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:49.865216 5544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:49.865216 5538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:49.865579 5531 server_base.cc:1034] running on GCE node
I20260808 22:30:49.865926 5531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:30:49.866533 5531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:30:49.867697 5531 hybrid_clock.cc:648] HybridClock initialized: now 1786228249867681 us; error 27 us; skew 500 ppm
I20260808 22:30:49.869433 5531 webserver.cc:460] Webserver started at http://127.5.11.195:44743/ using document root <none> and password file <none>
I20260808 22:30:49.869989 5531 fs_manager.cc:359] Metadata directory not provided
I20260808 22:30:49.870064 5531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:30:49.870297 5531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:30:49.872464 5531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-2/data/instance:
uuid: "fe71c637911f47f8a0d93040b4fa4860"
format_stamp: "Formatted at 2026-08-08 22:30:49 on dist-test-slave-7f01"
server_key: "63f77f038583143232857ce6474d5335"
server_key_iv: "0e5d554f12ca616b2ca5434c0c183e0c"
server_key_version: "encryptionkey@0"
I20260808 22:30:49.873009 5531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance:
uuid: "fe71c637911f47f8a0d93040b4fa4860"
format_stamp: "Formatted at 2026-08-08 22:30:49 on dist-test-slave-7f01"
server_key: "63f77f038583143232857ce6474d5335"
server_key_iv: "0e5d554f12ca616b2ca5434c0c183e0c"
server_key_version: "encryptionkey@0"
I20260808 22:30:49.876607 5531 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:30:49.879823 5551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:49.881346 5531 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260808 22:30:49.881467 5531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-2/wal
uuid: "fe71c637911f47f8a0d93040b4fa4860"
format_stamp: "Formatted at 2026-08-08 22:30:49 on dist-test-slave-7f01"
server_key: "63f77f038583143232857ce6474d5335"
server_key_iv: "0e5d554f12ca616b2ca5434c0c183e0c"
server_key_version: "encryptionkey@0"
I20260808 22:30:49.881592 5531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:30:49.906625 5531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:30:49.907230 5531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:30:49.907388 5531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:30:49.908004 5531 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260808 22:30:49.908984 5531 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260808 22:30:49.909035 5531 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:49.909075 5531 ts_tablet_manager.cc:610] Registered 0 tablets
I20260808 22:30:49.909091 5531 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:49.918128 5531 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.195:45245
I20260808 22:30:49.918141 5761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.195:45245 every 8 connection(s)
I20260808 22:30:49.919028 5531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-2/data/info.pb
I20260808 22:30:49.927886 5018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:49.920862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:35675 (local address 127.5.11.254:35993)
0808 22:30:49.921031 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:49.921035 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:49.922050 (+ 1015us) server_negotiation.cc:407] Connection header received
0808 22:30:49.923386 (+ 1336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:49.923388 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:49.923436 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:49.923513 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:49.924391 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:49.924934 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:49.926012 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:49.926220 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:49.926295 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:49.927427 (+ 1132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:49.927430 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:49.927432 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:30:49.927443 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:49.927460 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:30:49.927466 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:49.927522 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:49.927657 (+ 135us) server_negotiation.cc:299] Negotiation successful
0808 22:30:49.927707 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 22:30:49.928390 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 5531
I20260808 22:30:49.928520 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-2/wal/instance
I20260808 22:30:49.928779 5767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:49.921251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:35993 (local address 127.5.11.195:35675)
0808 22:30:49.921901 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:49.921949 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:49.923193 (+ 1244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:49.923679 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:49.923686 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:49.923783 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:30:49.924267 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:49.924280 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:49.925046 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:49.925049 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:49.925902 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:49.925921 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:49.926400 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:49.926404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:49.926552 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:49.927191 (+ 639us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0808 22:30:49.927192 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:30:49.927234 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0808 22:30:49.927300 (+ 66us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:30:49.927539 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:30:49.927545 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:30:49.927550 (+ 5us) client_negotiation.cc:769] Sending connection context
0808 22:30:49.927643 (+ 93us) client_negotiation.cc:240] Negotiation successful
0808 22:30:49.927902 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":128,"threads_started":1}
I20260808 22:30:49.928805 5167 external_mini_cluster.cc:1428] Setting key 49dd5529afa93e1818af56cc6d67791f
I20260808 22:30:49.929966 5763 heartbeater.cc:347] Connected to a master server at 127.5.11.254:35993
I20260808 22:30:49.930228 5763 heartbeater.cc:464] Registering TS with master...
I20260808 22:30:49.930785 5763 heartbeater.cc:511] Master 127.5.11.254:35993 requested a full tablet report, sending...
I20260808 22:30:49.931842 4939 ts_manager.cc:194] Registered new tserver with Master: fe71c637911f47f8a0d93040b4fa4860 (127.5.11.195:45245)
I20260808 22:30:49.932437 4939 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.5.11.195:35675
I20260808 22:30:49.942691 5167 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260808 22:30:49.948598 5018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:49.944760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38906 (local address 127.5.11.254:35993)
0808 22:30:49.944921 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:49.944928 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:49.945093 (+ 165us) server_negotiation.cc:407] Connection header received
0808 22:30:49.945250 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:49.945255 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:49.945300 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:49.945384 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:49.945660 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:49.946171 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:49.947040 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:49.947252 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:49.947359 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:49.948163 (+ 804us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:49.948167 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:49.948170 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:30:49.948183 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:49.948207 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:30:49.948215 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:49.948287 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:49.948405 (+ 118us) server_negotiation.cc:299] Negotiation successful
0808 22:30:49.948456 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260808 22:30:49.951359 4939 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38906:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260808 22:30:49.954452 4939 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260808 22:30:49.969204 5784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:49.963037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.195:45245 (local address 127.0.0.1:41848)
0808 22:30:49.963944 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:49.963999 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:49.964138 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:49.964960 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:49.964967 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:49.964988 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:30:49.965132 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:49.965139 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:49.966647 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:49.966650 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:49.967835 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:49.967844 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:49.968625 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:49.968629 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:49.968735 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:49.968774 (+ 39us) client_negotiation.cc:769] Sending connection context
0808 22:30:49.968908 (+ 134us) client_negotiation.cc:240] Negotiation successful
0808 22:30:49.969030 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":723,"mutex_wait_us":2,"thread_start_us":72,"threads_started":1}
I20260808 22:30:49.969373 5788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:49.963196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41848 (local address 127.5.11.195:45245)
0808 22:30:49.964514 (+ 1318us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:49.964529 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:49.964551 (+ 22us) server_negotiation.cc:407] Connection header received
0808 22:30:49.964630 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:49.964636 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:49.964806 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:49.964969 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:30:49.965245 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:49.966510 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:49.967958 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:49.968495 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:49.968624 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:49.968830 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:49.969082 (+ 252us) server_negotiation.cc:299] Negotiation successful
0808 22:30:49.969189 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1215,"thread_start_us":128,"threads_started":1}
I20260808 22:30:49.969560 5791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:49.963893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45904 (local address 127.5.11.194:40071)
0808 22:30:49.964351 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:49.964365 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:49.964712 (+ 347us) server_negotiation.cc:407] Connection header received
0808 22:30:49.964788 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:49.964793 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:49.964961 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:49.965088 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:30:49.965472 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:49.966188 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:49.967832 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:49.968655 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:49.968779 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:49.968988 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:49.969280 (+ 292us) server_negotiation.cc:299] Negotiation successful
0808 22:30:49.969403 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":130,"threads_started":1}
I20260808 22:30:49.969501 5786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:49.963219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:33527 (local address 127.0.0.1:33852)
0808 22:30:49.964141 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:49.964155 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:49.964250 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:49.965139 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:49.965143 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:49.965160 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:30:49.965318 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:49.965325 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:49.966744 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:49.966748 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:49.967849 (+ 1101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:49.967861 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:49.969067 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:49.969070 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:49.969168 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:49.969191 (+ 23us) client_negotiation.cc:769] Sending connection context
0808 22:30:49.969280 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 22:30:49.969348 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":862,"mutex_wait_us":51,"thread_start_us":120,"threads_started":1}
I20260808 22:30:49.969893 5792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:49.963897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33852 (local address 127.5.11.193:33527)
0808 22:30:49.964451 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:49.964464 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:49.964493 (+ 29us) server_negotiation.cc:407] Connection header received
0808 22:30:49.964582 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:49.964591 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:49.964808 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:49.964974 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:30:49.965442 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:49.966594 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:49.967994 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:49.968760 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:49.968853 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:49.969000 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:49.969599 (+ 599us) server_negotiation.cc:299] Negotiation successful
0808 22:30:49.969713 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":160,"threads_started":1}
I20260808 22:30:49.969657 5790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:49.963468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:40071 (local address 127.0.0.1:45904)
0808 22:30:49.964530 (+ 1062us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:49.964542 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:49.964626 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:49.965221 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:49.965223 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:49.965240 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:30:49.965331 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:49.965346 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:49.966315 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:49.966319 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:49.967676 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:49.967687 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:49.968930 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:49.968933 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:49.969012 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:49.969031 (+ 19us) client_negotiation.cc:769] Sending connection context
0808 22:30:49.969099 (+ 68us) client_negotiation.cc:240] Negotiation successful
0808 22:30:49.969153 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1004,"mutex_wait_us":7,"thread_start_us":93,"threads_started":1}
I20260808 22:30:49.972977 5648 tablet_service.cc:1467] Processing CreateTablet for tablet 02cd549223c14d7ca52621d27fcf67a8 (DEFAULT_TABLE table=test-table [id=f55e1fc72ced41e0a2cc45ff6025fda9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:30:49.973040 5175 tablet_service.cc:1467] Processing CreateTablet for tablet 02cd549223c14d7ca52621d27fcf67a8 (DEFAULT_TABLE table=test-table [id=f55e1fc72ced41e0a2cc45ff6025fda9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:30:49.973258 5404 tablet_service.cc:1467] Processing CreateTablet for tablet 02cd549223c14d7ca52621d27fcf67a8 (DEFAULT_TABLE table=test-table [id=f55e1fc72ced41e0a2cc45ff6025fda9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260808 22:30:49.974148 5404 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02cd549223c14d7ca52621d27fcf67a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:30:49.974148 5175 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02cd549223c14d7ca52621d27fcf67a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:30:49.974150 5648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 02cd549223c14d7ca52621d27fcf67a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:30:49.980437 5793 tablet_bootstrap.cc:492] T 02cd549223c14d7ca52621d27fcf67a8 P fe71c637911f47f8a0d93040b4fa4860: Bootstrap starting.
I20260808 22:30:49.981971 5796 tablet_bootstrap.cc:492] T 02cd549223c14d7ca52621d27fcf67a8 P ba1b9107fe864100a1e8cb108ddc0f3c: Bootstrap starting.
I20260808 22:30:49.982744 5793 tablet_bootstrap.cc:654] T 02cd549223c14d7ca52621d27fcf67a8 P fe71c637911f47f8a0d93040b4fa4860: Neither blocks nor log segments found. Creating new log.
I20260808 22:30:49.983471 5793 log.cc:824] T 02cd549223c14d7ca52621d27fcf67a8 P fe71c637911f47f8a0d93040b4fa4860: Log is configured to *not* fsync() on all Append() calls
I20260808 22:30:49.983804 5796 tablet_bootstrap.cc:654] T 02cd549223c14d7ca52621d27fcf67a8 P ba1b9107fe864100a1e8cb108ddc0f3c: Neither blocks nor log segments found. Creating new log.
I20260808 22:30:49.984109 5795 tablet_bootstrap.cc:492] T 02cd549223c14d7ca52621d27fcf67a8 P 7072be4f945e4694bfc9566773b28c0e: Bootstrap starting.
I20260808 22:30:49.984660 5796 log.cc:824] T 02cd549223c14d7ca52621d27fcf67a8 P ba1b9107fe864100a1e8cb108ddc0f3c: Log is configured to *not* fsync() on all Append() calls
I20260808 22:30:49.985778 5793 tablet_bootstrap.cc:492] T 02cd549223c14d7ca52621d27fcf67a8 P fe71c637911f47f8a0d93040b4fa4860: No bootstrap required, opened a new log
I20260808 22:30:49.985978 5793 ts_tablet_manager.cc:1397] T 02cd549223c14d7ca52621d27fcf67a8 P fe71c637911f47f8a0d93040b4fa4860: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260808 22:30:49.986349 5795 tablet_bootstrap.cc:654] T 02cd549223c14d7ca52621d27fcf67a8 P 7072be4f945e4694bfc9566773b28c0e: Neither blocks nor log segments found. Creating new log.
I20260808 22:30:49.987020 5795 log.cc:824] T 02cd549223c14d7ca52621d27fcf67a8 P 7072be4f945e4694bfc9566773b28c0e: Log is configured to *not* fsync() on all Append() calls
I20260808 22:30:49.987095 5796 tablet_bootstrap.cc:492] T 02cd549223c14d7ca52621d27fcf67a8 P ba1b9107fe864100a1e8cb108ddc0f3c: No bootstrap required, opened a new log
I20260808 22:30:49.987282 5796 ts_tablet_manager.cc:1397] T 02cd549223c14d7ca52621d27fcf67a8 P ba1b9107fe864100a1e8cb108ddc0f3c: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260808 22:30:49.989009 5795 tablet_bootstrap.cc:492] T 02cd549223c14d7ca52621d27fcf67a8 P 7072be4f945e4694bfc9566773b28c0e: No bootstrap required, opened a new log
I20260808 22:30:49.989203 5795 ts_tablet_manager.cc:1397] T 02cd549223c14d7ca52621d27fcf67a8 P 7072be4f945e4694bfc9566773b28c0e: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260808 22:30:49.990581 5793 raft_consensus.cc:348] T 02cd549223c14d7ca52621d27fcf67a8 P fe71c637911f47f8a0d93040b4fa4860 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe71c637911f47f8a0d93040b4fa4860" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 45245 } } peers { permanent_uuid: "7072be4f945e4694bfc9566773b28c0e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 33527 } } peers { permanent_uuid: "ba1b9107fe864100a1e8cb108ddc0f3c" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 40071 } }
I20260808 22:30:49.990861 5793 raft_consensus.cc:374] T 02cd549223c14d7ca52621d27fcf67a8 P fe71c637911f47f8a0d93040b4fa4860 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:30:49.990932 5793 raft_consensus.cc:729] T 02cd549223c14d7ca52621d27fcf67a8 P fe71c637911f47f8a0d93040b4fa4860 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fe71c637911f47f8a0d93040b4fa4860, State: Initialized, Role: FOLLOWER
I20260808 22:30:49.991403 5793 consensus_queue.cc:260] T 02cd549223c14d7ca52621d27fcf67a8 P fe71c637911f47f8a0d93040b4fa4860 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe71c637911f47f8a0d93040b4fa4860" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 45245 } } peers { permanent_uuid: "7072be4f945e4694bfc9566773b28c0e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 33527 } } peers { permanent_uuid: "ba1b9107fe864100a1e8cb108ddc0f3c" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 40071 } }
I20260808 22:30:49.991446 5796 raft_consensus.cc:348] T 02cd549223c14d7ca52621d27fcf67a8 P ba1b9107fe864100a1e8cb108ddc0f3c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe71c637911f47f8a0d93040b4fa4860" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 45245 } } peers { permanent_uuid: "7072be4f945e4694bfc9566773b28c0e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 33527 } } peers { permanent_uuid: "ba1b9107fe864100a1e8cb108ddc0f3c" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 40071 } }
I20260808 22:30:49.991680 5796 raft_consensus.cc:374] T 02cd549223c14d7ca52621d27fcf67a8 P ba1b9107fe864100a1e8cb108ddc0f3c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:30:49.991753 5796 raft_consensus.cc:729] T 02cd549223c14d7ca52621d27fcf67a8 P ba1b9107fe864100a1e8cb108ddc0f3c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba1b9107fe864100a1e8cb108ddc0f3c, State: Initialized, Role: FOLLOWER
I20260808 22:30:49.992189 5796 consensus_queue.cc:260] T 02cd549223c14d7ca52621d27fcf67a8 P ba1b9107fe864100a1e8cb108ddc0f3c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe71c637911f47f8a0d93040b4fa4860" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 45245 } } peers { permanent_uuid: "7072be4f945e4694bfc9566773b28c0e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 33527 } } peers { permanent_uuid: "ba1b9107fe864100a1e8cb108ddc0f3c" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 40071 } }
I20260808 22:30:49.992524 5763 heartbeater.cc:503] Master 127.5.11.254:35993 was elected leader, sending a full tablet report...
I20260808 22:30:49.992677 5793 ts_tablet_manager.cc:1428] T 02cd549223c14d7ca52621d27fcf67a8 P fe71c637911f47f8a0d93040b4fa4860: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260808 22:30:49.992767 5795 raft_consensus.cc:348] T 02cd549223c14d7ca52621d27fcf67a8 P 7072be4f945e4694bfc9566773b28c0e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe71c637911f47f8a0d93040b4fa4860" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 45245 } } peers { permanent_uuid: "7072be4f945e4694bfc9566773b28c0e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 33527 } } peers { permanent_uuid: "ba1b9107fe864100a1e8cb108ddc0f3c" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 40071 } }
I20260808 22:30:49.992996 5795 raft_consensus.cc:374] T 02cd549223c14d7ca52621d27fcf67a8 P 7072be4f945e4694bfc9566773b28c0e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:30:49.993052 5795 raft_consensus.cc:729] T 02cd549223c14d7ca52621d27fcf67a8 P 7072be4f945e4694bfc9566773b28c0e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7072be4f945e4694bfc9566773b28c0e, State: Initialized, Role: FOLLOWER
I20260808 22:30:49.993116 5796 ts_tablet_manager.cc:1428] T 02cd549223c14d7ca52621d27fcf67a8 P ba1b9107fe864100a1e8cb108ddc0f3c: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
I20260808 22:30:49.993188 5526 heartbeater.cc:503] Master 127.5.11.254:35993 was elected leader, sending a full tablet report...
I20260808 22:30:49.993441 5795 consensus_queue.cc:260] T 02cd549223c14d7ca52621d27fcf67a8 P 7072be4f945e4694bfc9566773b28c0e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe71c637911f47f8a0d93040b4fa4860" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 45245 } } peers { permanent_uuid: "7072be4f945e4694bfc9566773b28c0e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 33527 } } peers { permanent_uuid: "ba1b9107fe864100a1e8cb108ddc0f3c" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 40071 } }
I20260808 22:30:49.994345 5795 ts_tablet_manager.cc:1428] T 02cd549223c14d7ca52621d27fcf67a8 P 7072be4f945e4694bfc9566773b28c0e: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260808 22:30:49.994680 5287 heartbeater.cc:503] Master 127.5.11.254:35993 was elected leader, sending a full tablet report...
W20260808 22:30:50.034564 5289 tablet.cc:2369] T 02cd549223c14d7ca52621d27fcf67a8 P 7072be4f945e4694bfc9566773b28c0e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:30:50.100004 5800 raft_consensus.cc:482] T 02cd549223c14d7ca52621d27fcf67a8 P fe71c637911f47f8a0d93040b4fa4860 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260808 22:30:50.100224 5800 raft_consensus.cc:504] T 02cd549223c14d7ca52621d27fcf67a8 P fe71c637911f47f8a0d93040b4fa4860 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe71c637911f47f8a0d93040b4fa4860" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 45245 } } peers { permanent_uuid: "7072be4f945e4694bfc9566773b28c0e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 33527 } } peers { permanent_uuid: "ba1b9107fe864100a1e8cb108ddc0f3c" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 40071 } }
I20260808 22:30:50.101432 5800 leader_election.cc:288] T 02cd549223c14d7ca52621d27fcf67a8 P fe71c637911f47f8a0d93040b4fa4860 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7072be4f945e4694bfc9566773b28c0e (127.5.11.193:33527), ba1b9107fe864100a1e8cb108ddc0f3c (127.5.11.194:40071)
I20260808 22:30:50.105448 5792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:50.101760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:36341 (local address 127.5.11.193:33527)
0808 22:30:50.101906 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:50.101911 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:50.101928 (+ 17us) server_negotiation.cc:407] Connection header received
0808 22:30:50.102107 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:50.102111 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:50.102172 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:50.102280 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:30:50.102557 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:50.103058 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:50.104357 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:50.104936 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:50.105033 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:50.105074 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:50.105287 (+ 213us) server_negotiation.cc:299] Negotiation successful
0808 22:30:50.105341 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 22:30:50.105473 5767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:50.101634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:33527 (local address 127.5.11.195:36341)
0808 22:30:50.101839 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:50.101857 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:50.101960 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:50.102295 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:50.102298 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:50.102316 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:30:50.102434 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:50.102440 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:50.103200 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:50.103203 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:50.104235 (+ 1032us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:50.104245 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:50.105054 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:50.105058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:50.105158 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:50.105185 (+ 27us) client_negotiation.cc:769] Sending connection context
0808 22:30:50.105263 (+ 78us) client_negotiation.cc:240] Negotiation successful
0808 22:30:50.105329 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":1}
I20260808 22:30:50.106030 5805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:50.101773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.194:40071 (local address 127.5.11.195:57401)
0808 22:30:50.102134 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:50.102145 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:30:50.102274 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:30:50.102577 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:30:50.102580 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:30:50.102597 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0808 22:30:50.102721 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:50.102726 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:50.103696 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:50.103699 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:50.104729 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:30:50.104741 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:30:50.105584 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:30:50.105588 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:30:50.105696 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:50.105718 (+ 22us) client_negotiation.cc:769] Sending connection context
0808 22:30:50.105835 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 22:30:50.105898 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"mutex_wait_us":22,"thread_start_us":145,"threads_started":1}
I20260808 22:30:50.106077 5791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:50.101825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.5.11.195:57401 (local address 127.5.11.194:40071)
0808 22:30:50.101966 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:50.101971 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:50.102230 (+ 259us) server_negotiation.cc:407] Connection header received
0808 22:30:50.102416 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:50.102420 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:50.102472 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:50.102555 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0808 22:30:50.102841 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:50.103580 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:50.104863 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:50.105432 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:50.105554 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:50.105603 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:50.105881 (+ 278us) server_negotiation.cc:299] Negotiation successful
0808 22:30:50.105967 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 22:30:50.106195 5211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02cd549223c14d7ca52621d27fcf67a8" candidate_uuid: "fe71c637911f47f8a0d93040b4fa4860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7072be4f945e4694bfc9566773b28c0e" is_pre_election: true
I20260808 22:30:50.106524 5211 raft_consensus.cc:2454] T 02cd549223c14d7ca52621d27fcf67a8 P 7072be4f945e4694bfc9566773b28c0e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe71c637911f47f8a0d93040b4fa4860 in term 0.
I20260808 22:30:50.106645 5435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02cd549223c14d7ca52621d27fcf67a8" candidate_uuid: "fe71c637911f47f8a0d93040b4fa4860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba1b9107fe864100a1e8cb108ddc0f3c" is_pre_election: true
I20260808 22:30:50.106940 5435 raft_consensus.cc:2454] T 02cd549223c14d7ca52621d27fcf67a8 P ba1b9107fe864100a1e8cb108ddc0f3c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fe71c637911f47f8a0d93040b4fa4860 in term 0.
I20260808 22:30:50.107044 5557 leader_election.cc:302] T 02cd549223c14d7ca52621d27fcf67a8 P fe71c637911f47f8a0d93040b4fa4860 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7072be4f945e4694bfc9566773b28c0e, fe71c637911f47f8a0d93040b4fa4860; no voters:
I20260808 22:30:50.107376 5800 raft_consensus.cc:2781] T 02cd549223c14d7ca52621d27fcf67a8 P fe71c637911f47f8a0d93040b4fa4860 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260808 22:30:50.107483 5800 raft_consensus.cc:482] T 02cd549223c14d7ca52621d27fcf67a8 P fe71c637911f47f8a0d93040b4fa4860 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260808 22:30:50.107530 5800 raft_consensus.cc:3037] T 02cd549223c14d7ca52621d27fcf67a8 P fe71c637911f47f8a0d93040b4fa4860 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:30:50.108629 5800 raft_consensus.cc:504] T 02cd549223c14d7ca52621d27fcf67a8 P fe71c637911f47f8a0d93040b4fa4860 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe71c637911f47f8a0d93040b4fa4860" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 45245 } } peers { permanent_uuid: "7072be4f945e4694bfc9566773b28c0e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 33527 } } peers { permanent_uuid: "ba1b9107fe864100a1e8cb108ddc0f3c" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 40071 } }
I20260808 22:30:50.108942 5800 leader_election.cc:288] T 02cd549223c14d7ca52621d27fcf67a8 P fe71c637911f47f8a0d93040b4fa4860 [CANDIDATE]: Term 1 election: Requested vote from peers 7072be4f945e4694bfc9566773b28c0e (127.5.11.193:33527), ba1b9107fe864100a1e8cb108ddc0f3c (127.5.11.194:40071)
I20260808 22:30:50.109295 5211 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02cd549223c14d7ca52621d27fcf67a8" candidate_uuid: "fe71c637911f47f8a0d93040b4fa4860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7072be4f945e4694bfc9566773b28c0e"
I20260808 22:30:50.109336 5435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "02cd549223c14d7ca52621d27fcf67a8" candidate_uuid: "fe71c637911f47f8a0d93040b4fa4860" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba1b9107fe864100a1e8cb108ddc0f3c"
I20260808 22:30:50.109427 5211 raft_consensus.cc:3037] T 02cd549223c14d7ca52621d27fcf67a8 P 7072be4f945e4694bfc9566773b28c0e [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:30:50.109467 5435 raft_consensus.cc:3037] T 02cd549223c14d7ca52621d27fcf67a8 P ba1b9107fe864100a1e8cb108ddc0f3c [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:30:50.110525 5211 raft_consensus.cc:2454] T 02cd549223c14d7ca52621d27fcf67a8 P 7072be4f945e4694bfc9566773b28c0e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe71c637911f47f8a0d93040b4fa4860 in term 1.
I20260808 22:30:50.110639 5435 raft_consensus.cc:2454] T 02cd549223c14d7ca52621d27fcf67a8 P ba1b9107fe864100a1e8cb108ddc0f3c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fe71c637911f47f8a0d93040b4fa4860 in term 1.
I20260808 22:30:50.110880 5557 leader_election.cc:302] T 02cd549223c14d7ca52621d27fcf67a8 P fe71c637911f47f8a0d93040b4fa4860 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7072be4f945e4694bfc9566773b28c0e, fe71c637911f47f8a0d93040b4fa4860; no voters:
I20260808 22:30:50.111063 5800 raft_consensus.cc:2781] T 02cd549223c14d7ca52621d27fcf67a8 P fe71c637911f47f8a0d93040b4fa4860 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:30:50.111296 5800 raft_consensus.cc:686] T 02cd549223c14d7ca52621d27fcf67a8 P fe71c637911f47f8a0d93040b4fa4860 [term 1 LEADER]: Becoming Leader. State: Replica: fe71c637911f47f8a0d93040b4fa4860, State: Running, Role: LEADER
I20260808 22:30:50.111654 5800 consensus_queue.cc:237] T 02cd549223c14d7ca52621d27fcf67a8 P fe71c637911f47f8a0d93040b4fa4860 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe71c637911f47f8a0d93040b4fa4860" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 45245 } } peers { permanent_uuid: "7072be4f945e4694bfc9566773b28c0e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 33527 } } peers { permanent_uuid: "ba1b9107fe864100a1e8cb108ddc0f3c" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 40071 } }
I20260808 22:30:50.114847 4941 catalog_manager.cc:5495] T 02cd549223c14d7ca52621d27fcf67a8 P fe71c637911f47f8a0d93040b4fa4860 reported cstate change: term changed from 0 to 1, leader changed from <none> to fe71c637911f47f8a0d93040b4fa4860 (127.5.11.195). New cstate: current_term: 1 leader_uuid: "fe71c637911f47f8a0d93040b4fa4860" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fe71c637911f47f8a0d93040b4fa4860" member_type: VOTER last_known_addr { host: "127.5.11.195" port: 45245 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7072be4f945e4694bfc9566773b28c0e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 33527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ba1b9107fe864100a1e8cb108ddc0f3c" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 40071 } health_report { overall_health: UNKNOWN } } }
I20260808 22:30:50.157245 5788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:50.153679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41850 (local address 127.5.11.195:45245)
0808 22:30:50.153833 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:50.153839 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:50.153858 (+ 19us) server_negotiation.cc:407] Connection header received
0808 22:30:50.153951 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:50.153954 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:50.154019 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:50.154136 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:30:50.154380 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:50.154879 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:50.155777 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:50.155993 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:50.156108 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:50.156420 (+ 312us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:30:50.156844 (+ 424us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:30:50.156993 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:50.157039 (+ 46us) server_negotiation.cc:299] Negotiation successful
0808 22:30:50.157092 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260808 22:30:50.163184 5435 raft_consensus.cc:1261] T 02cd549223c14d7ca52621d27fcf67a8 P ba1b9107fe864100a1e8cb108ddc0f3c [term 1 FOLLOWER]: Refusing update from remote peer fe71c637911f47f8a0d93040b4fa4860: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:30:50.163184 5211 raft_consensus.cc:1261] T 02cd549223c14d7ca52621d27fcf67a8 P 7072be4f945e4694bfc9566773b28c0e [term 1 FOLLOWER]: Refusing update from remote peer fe71c637911f47f8a0d93040b4fa4860: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:30:50.164001 5800 consensus_queue.cc:1035] T 02cd549223c14d7ca52621d27fcf67a8 P fe71c637911f47f8a0d93040b4fa4860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba1b9107fe864100a1e8cb108ddc0f3c" member_type: VOTER last_known_addr { host: "127.5.11.194" port: 40071 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:30:50.164201 5808 consensus_queue.cc:1035] T 02cd549223c14d7ca52621d27fcf67a8 P fe71c637911f47f8a0d93040b4fa4860 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7072be4f945e4694bfc9566773b28c0e" member_type: VOTER last_known_addr { host: "127.5.11.193" port: 33527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260808 22:30:50.169960 5765 tablet.cc:2369] T 02cd549223c14d7ca52621d27fcf67a8 P fe71c637911f47f8a0d93040b4fa4860: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260808 22:30:50.170661 5818 mvcc.cc:205] Tried to move back new op lower bound from 7316390912660869120 to 7316390912460378112. Current Snapshot: MvccSnapshot[applied={T|T < 7316390912660869120}]
I20260808 22:30:50.173736 5819 mvcc.cc:205] Tried to move back new op lower bound from 7316390912660869120 to 7316390912460378112. Current Snapshot: MvccSnapshot[applied={T|T < 7316390912660869120}]
I20260808 22:30:50.179713 4941 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38906:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:30:50.179935 4941 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38906:
table { table_name: "test-table" } modify_external_catalogs: true
I20260808 22:30:50.182194 4941 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b26dc241dcd34c60b584eb64ede93f8a: Sending DeleteTablet for 3 replicas of tablet 02cd549223c14d7ca52621d27fcf67a8
I20260808 22:30:50.182915 5175 tablet_service.cc:1514] Processing DeleteTablet for tablet 02cd549223c14d7ca52621d27fcf67a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:30:50 UTC) from {username='slave'} at 127.0.0.1:33852
I20260808 22:30:50.183018 5404 tablet_service.cc:1514] Processing DeleteTablet for tablet 02cd549223c14d7ca52621d27fcf67a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:30:50 UTC) from {username='slave'} at 127.0.0.1:45904
I20260808 22:30:50.183602 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 5058
I20260808 22:30:50.183648 5648 tablet_service.cc:1514] Processing DeleteTablet for tablet 02cd549223c14d7ca52621d27fcf67a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-08 22:30:50 UTC) from {username='slave'} at 127.0.0.1:41848
I20260808 22:30:50.184355 5837 tablet_replica.cc:330] T 02cd549223c14d7ca52621d27fcf67a8 P fe71c637911f47f8a0d93040b4fa4860: stopping tablet replica
I20260808 22:30:50.184659 5837 raft_consensus.cc:2229] T 02cd549223c14d7ca52621d27fcf67a8 P fe71c637911f47f8a0d93040b4fa4860 [term 1 LEADER]: Raft consensus shutting down.
I20260808 22:30:50.184995 5837 raft_consensus.cc:2258] T 02cd549223c14d7ca52621d27fcf67a8 P fe71c637911f47f8a0d93040b4fa4860 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:30:50.187373 5837 ts_tablet_manager.cc:1905] T 02cd549223c14d7ca52621d27fcf67a8 P fe71c637911f47f8a0d93040b4fa4860: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:30:50.189929 5837 ts_tablet_manager.cc:1918] T 02cd549223c14d7ca52621d27fcf67a8 P fe71c637911f47f8a0d93040b4fa4860: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:30:50.190054 5837 log.cc:1196] T 02cd549223c14d7ca52621d27fcf67a8 P fe71c637911f47f8a0d93040b4fa4860: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-2/wal/wals/02cd549223c14d7ca52621d27fcf67a8
I20260808 22:30:50.190347 5837 ts_tablet_manager.cc:1939] T 02cd549223c14d7ca52621d27fcf67a8 P fe71c637911f47f8a0d93040b4fa4860: Deleting consensus metadata
I20260808 22:30:50.195927 5836 tablet_replica.cc:330] T 02cd549223c14d7ca52621d27fcf67a8 P ba1b9107fe864100a1e8cb108ddc0f3c: stopping tablet replica
I20260808 22:30:50.196357 5836 raft_consensus.cc:2229] T 02cd549223c14d7ca52621d27fcf67a8 P ba1b9107fe864100a1e8cb108ddc0f3c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260808 22:30:50.196754 5836 raft_consensus.cc:2258] T 02cd549223c14d7ca52621d27fcf67a8 P ba1b9107fe864100a1e8cb108ddc0f3c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260808 22:30:50.197863 4917 catalog_manager.cc:4841] TS fe71c637911f47f8a0d93040b4fa4860 (127.5.11.195:45245): tablet 02cd549223c14d7ca52621d27fcf67a8 (table test-table [id=f55e1fc72ced41e0a2cc45ff6025fda9]) successfully deleted
I20260808 22:30:50.199277 5836 ts_tablet_manager.cc:1905] T 02cd549223c14d7ca52621d27fcf67a8 P ba1b9107fe864100a1e8cb108ddc0f3c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260808 22:30:50.202761 5836 ts_tablet_manager.cc:1918] T 02cd549223c14d7ca52621d27fcf67a8 P ba1b9107fe864100a1e8cb108ddc0f3c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260808 22:30:50.202890 5836 log.cc:1196] T 02cd549223c14d7ca52621d27fcf67a8 P ba1b9107fe864100a1e8cb108ddc0f3c: Deleting WAL directory at /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786227991988852-5167-0/minicluster-data/ts-1/wal/wals/02cd549223c14d7ca52621d27fcf67a8
I20260808 22:30:50.203246 5836 ts_tablet_manager.cc:1939] T 02cd549223c14d7ca52621d27fcf67a8 P ba1b9107fe864100a1e8cb108ddc0f3c: Deleting consensus metadata
I20260808 22:30:50.204581 4916 catalog_manager.cc:4841] TS ba1b9107fe864100a1e8cb108ddc0f3c (127.5.11.194:40071): tablet 02cd549223c14d7ca52621d27fcf67a8 (table test-table [id=f55e1fc72ced41e0a2cc45ff6025fda9]) successfully deleted
W20260808 22:30:50.205534 4920 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.5.11.193:33527 (error 108)
I20260808 22:30:50.206351 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 5291
I20260808 22:30:50.206633 5790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:50.206308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.193:33527 (local address 127.0.0.1:33860)
0808 22:30:50.206448 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0808 22:30:50.206535 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.193:33527: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260808 22:30:50.209072 4920 catalog_manager.cc:4584] TS 7072be4f945e4694bfc9566773b28c0e (127.5.11.193:33527): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 02cd549223c14d7ca52621d27fcf67a8: Network error: Client connection negotiation failed: client connection to 127.5.11.193:33527: connect: Connection refused (error 111)
I20260808 22:30:50.215893 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 5531
I20260808 22:30:50.225358 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 4892
2026-08-08T22:30:50Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1139 ms)
[----------] 6 tests from EncryptionPolicyTest (10830 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:30:50 dist-test-slave-7f01 krb5kdc[5843](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:30:50 dist-test-slave-7f01 krb5kdc[5843](info): set up 2 sockets
Aug 08 22:30:50 dist-test-slave-7f01 krb5kdc[5843](info): commencing operation
krb5kdc: starting...
W20260808 22:30:51.491503 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:30:51 dist-test-slave-7f01 krb5kdc[5843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:30:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:30:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 22:30:51.620110 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40363
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35059
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40363
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:30:51.731523 5871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:30:51.731838 5871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:30:51.731896 5871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:30:51.735951 5871 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:30:51.736006 5871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:30:51.736025 5871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:30:51.736042 5871 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:30:51.736057 5871 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:30:51.741164 5871 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35059
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40363
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5871
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:30:51.742115 5871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:30:51.743052 5871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:30:51.751475 5876 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:51.751508 5877 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:51.751485 5881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:51.751646 5871 server_base.cc:1034] running on GCE node
I20260808 22:30:51.752403 5871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:30:51.753471 5871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:30:51.754633 5871 hybrid_clock.cc:648] HybridClock initialized: now 1786228251754614 us; error 29 us; skew 500 ppm
Aug 08 22:30:51 dist-test-slave-7f01 krb5kdc[5843](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228251, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:30:51.757370 5871 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 22:30:51.758229 5871 webserver.cc:460] Webserver started at http://127.0.0.1:34857/ using document root <none> and password file <none>
I20260808 22:30:51.758788 5871 fs_manager.cc:359] Metadata directory not provided
I20260808 22:30:51.758850 5871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:30:51.759155 5871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:30:51.761533 5871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "82574fceed8f4a7db417647ce5e39866"
format_stamp: "Formatted at 2026-08-08 22:30:51 on dist-test-slave-7f01"
server_key: "55772c9db7aadc1394b898c8e2b8c66e"
server_key_iv: "df5bd3086a85ef720dbf6799989c0be5"
server_key_version: "encryptionkey@0"
I20260808 22:30:51.762079 5871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "82574fceed8f4a7db417647ce5e39866"
format_stamp: "Formatted at 2026-08-08 22:30:51 on dist-test-slave-7f01"
server_key: "55772c9db7aadc1394b898c8e2b8c66e"
server_key_iv: "df5bd3086a85ef720dbf6799989c0be5"
server_key_version: "encryptionkey@0"
I20260808 22:30:51.765729 5871 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260808 22:30:51.768070 5897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:51.769271 5871 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 22:30:51.769375 5871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "82574fceed8f4a7db417647ce5e39866"
format_stamp: "Formatted at 2026-08-08 22:30:51 on dist-test-slave-7f01"
server_key: "55772c9db7aadc1394b898c8e2b8c66e"
server_key_iv: "df5bd3086a85ef720dbf6799989c0be5"
server_key_version: "encryptionkey@0"
I20260808 22:30:51.769460 5871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:30:51.784549 5871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:30:51.787252 5871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:30:51.787403 5871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:30:51.794716 5871 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40363
I20260808 22:30:51.794744 5987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40363 every 8 connection(s)
I20260808 22:30:51.795673 5871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:30:51.796515 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 5871
I20260808 22:30:51.796690 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:30:51.796998 5167 external_mini_cluster.cc:1428] Setting key 7f5d06b79d80f639be92b2e2c892ec44
I20260808 22:30:51.798985 5989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:30:51.805737 5989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82574fceed8f4a7db417647ce5e39866: Bootstrap starting.
Aug 08 22:30:51 dist-test-slave-7f01 krb5kdc[5843](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228251, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 22:30:51.808408 5989 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82574fceed8f4a7db417647ce5e39866: Neither blocks nor log segments found. Creating new log.
I20260808 22:30:51.809132 5989 log.cc:824] T 00000000000000000000000000000000 P 82574fceed8f4a7db417647ce5e39866: Log is configured to *not* fsync() on all Append() calls
I20260808 22:30:51.811393 5989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82574fceed8f4a7db417647ce5e39866: No bootstrap required, opened a new log
I20260808 22:30:51.812294 5997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:51.798469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56075 (local address 127.0.0.1:40363)
0808 22:30:51.798896 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:51.798914 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:51.798957 (+ 43us) server_negotiation.cc:407] Connection header received
0808 22:30:51.799848 (+ 891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:51.799871 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:51.800299 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:51.800667 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:51.801369 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:51.802882 (+ 1513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:51.803769 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:51.804452 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:51.804627 (+ 175us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:51.807456 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:51.807461 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:51.807472 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:30:51.807499 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:51.809282 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:51.809843 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:51.809852 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:51.809861 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:51.810023 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:51.810371 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:51.810373 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:51.810375 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:51.810752 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:51.810915 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:51.811249 (+ 334us) server_negotiation.cc:299] Negotiation successful
0808 22:30:51.811497 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":281,"thread_start_us":133,"threads_started":1}
I20260808 22:30:51.815168 5167 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:30:51.814992 5989 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82574fceed8f4a7db417647ce5e39866 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82574fceed8f4a7db417647ce5e39866" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40363 } }
I20260808 22:30:51.815253 5989 raft_consensus.cc:374] T 00000000000000000000000000000000 P 82574fceed8f4a7db417647ce5e39866 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:30:51.815311 5989 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82574fceed8f4a7db417647ce5e39866 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82574fceed8f4a7db417647ce5e39866, State: Initialized, Role: FOLLOWER
I20260808 22:30:51.815800 5989 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82574fceed8f4a7db417647ce5e39866 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82574fceed8f4a7db417647ce5e39866" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40363 } }
I20260808 22:30:51.815976 5989 raft_consensus.cc:388] T 00000000000000000000000000000000 P 82574fceed8f4a7db417647ce5e39866 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:30:51.816051 5989 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82574fceed8f4a7db417647ce5e39866 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:30:51.816165 5989 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82574fceed8f4a7db417647ce5e39866 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:30:51.817315 5989 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82574fceed8f4a7db417647ce5e39866 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82574fceed8f4a7db417647ce5e39866" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40363 } }
I20260808 22:30:51.817714 5989 leader_election.cc:302] T 00000000000000000000000000000000 P 82574fceed8f4a7db417647ce5e39866 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 82574fceed8f4a7db417647ce5e39866; no voters:
I20260808 22:30:51.818099 5989 leader_election.cc:288] T 00000000000000000000000000000000 P 82574fceed8f4a7db417647ce5e39866 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:30:51.818161 6018 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82574fceed8f4a7db417647ce5e39866 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:30:51.818359 6018 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82574fceed8f4a7db417647ce5e39866 [term 1 LEADER]: Becoming Leader. State: Replica: 82574fceed8f4a7db417647ce5e39866, State: Running, Role: LEADER
I20260808 22:30:51.818661 6018 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82574fceed8f4a7db417647ce5e39866 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82574fceed8f4a7db417647ce5e39866" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40363 } }
I20260808 22:30:51.819339 5989 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82574fceed8f4a7db417647ce5e39866 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:30:51.819861 6018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82574fceed8f4a7db417647ce5e39866 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "82574fceed8f4a7db417647ce5e39866" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82574fceed8f4a7db417647ce5e39866" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40363 } } }
I20260808 22:30:51.819979 6018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82574fceed8f4a7db417647ce5e39866 [sys.catalog]: This master's current role is: LEADER
I20260808 22:30:51.820369 6025 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:30:51.820461 6019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82574fceed8f4a7db417647ce5e39866 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82574fceed8f4a7db417647ce5e39866. Latest consensus state: current_term: 1 leader_uuid: "82574fceed8f4a7db417647ce5e39866" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "82574fceed8f4a7db417647ce5e39866" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40363 } } }
I20260808 22:30:51.820623 6019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82574fceed8f4a7db417647ce5e39866 [sys.catalog]: This master's current role is: LEADER
I20260808 22:30:51.823822 6025 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:30:51.826818 5997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:51.817534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42213 (local address 127.0.0.1:40363)
0808 22:30:51.817645 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:51.817649 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:51.817879 (+ 230us) server_negotiation.cc:407] Connection header received
0808 22:30:51.817997 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:51.818000 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:51.818048 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:51.818137 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:51.819149 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:51.819882 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:51.821080 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:51.821419 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:51.821516 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:51.823208 (+ 1692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:51.823212 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:51.823214 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:30:51.823239 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:51.824767 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:51.825338 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:51.825341 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:51.825343 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:51.825443 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:51.825859 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:51.825862 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:51.825864 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:51.826081 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:51.826236 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:51.826557 (+ 321us) server_negotiation.cc:299] Negotiation successful
0808 22:30:51.826658 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260808 22:30:51.829411 6025 catalog_manager.cc:1347] Generated new cluster ID: 0ee998804543413483fdba6d43975b89
I20260808 22:30:51.829474 6025 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:30:51.842749 6025 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:30:51.843667 6025 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:30:51.847785 6025 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82574fceed8f4a7db417647ce5e39866: Generated new TSK 0
I20260808 22:30:51.848425 6025 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:30:51.855834 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 5871
W20260808 22:30:51.878328 5840 connection.cc:535] client connection to 127.0.0.1:40363 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40363): Connection reset by peer (error 104)
2026-08-08T22:30:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1640 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:30:51 dist-test-slave-7f01 krb5kdc[6051](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:30:51 dist-test-slave-7f01 krb5kdc[6051](info): set up 2 sockets
Aug 08 22:30:51 dist-test-slave-7f01 krb5kdc[6051](info): commencing operation
krb5kdc: starting...
W20260808 22:30:53.124986 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:30:53 dist-test-slave-7f01 krb5kdc[6051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228253, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:30:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:30:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 22:30:53.250283 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33217
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46325
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33217
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:30:53.360991 6076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:30:53.361284 6076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:30:53.361366 6076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:30:53.365275 6076 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:30:53.365365 6076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:30:53.365403 6076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:30:53.365463 6076 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:30:53.365504 6076 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:30:53.370612 6076 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46325
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33217
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:30:53.371661 6076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:30:53.372715 6076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:30:53.380973 6085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:53.381004 6087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:53.381004 6083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:53.381377 6076 server_base.cc:1034] running on GCE node
I20260808 22:30:53.381942 6076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:30:53.383021 6076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:30:53.384213 6076 hybrid_clock.cc:648] HybridClock initialized: now 1786228253384193 us; error 29 us; skew 500 ppm
Aug 08 22:30:53 dist-test-slave-7f01 krb5kdc[6051](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228253, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:30:53.387009 6076 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 22:30:53.387929 6076 webserver.cc:460] Webserver started at http://127.0.0.1:39943/ using document root <none> and password file <none>
I20260808 22:30:53.388496 6076 fs_manager.cc:359] Metadata directory not provided
I20260808 22:30:53.388589 6076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:30:53.388824 6076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:30:53.390848 6076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "58d73d1ede9740f7a11ebd34e01dfb9b"
format_stamp: "Formatted at 2026-08-08 22:30:53 on dist-test-slave-7f01"
server_key: "22becb1cd53dae765e01b9a2f6ce8fd7"
server_key_iv: "eb5272b4aeebc173525bd789560845f2"
server_key_version: "encryptionkey@0"
I20260808 22:30:53.391438 6076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "58d73d1ede9740f7a11ebd34e01dfb9b"
format_stamp: "Formatted at 2026-08-08 22:30:53 on dist-test-slave-7f01"
server_key: "22becb1cd53dae765e01b9a2f6ce8fd7"
server_key_iv: "eb5272b4aeebc173525bd789560845f2"
server_key_version: "encryptionkey@0"
I20260808 22:30:53.395113 6076 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 22:30:53.397706 6102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:53.398902 6076 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:30:53.399050 6076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "58d73d1ede9740f7a11ebd34e01dfb9b"
format_stamp: "Formatted at 2026-08-08 22:30:53 on dist-test-slave-7f01"
server_key: "22becb1cd53dae765e01b9a2f6ce8fd7"
server_key_iv: "eb5272b4aeebc173525bd789560845f2"
server_key_version: "encryptionkey@0"
I20260808 22:30:53.399163 6076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:30:53.421777 6076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:30:53.425138 6076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:30:53.425355 6076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:30:53.432417 6076 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33217
I20260808 22:30:53.432436 6188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33217 every 8 connection(s)
I20260808 22:30:53.433377 6076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:30:53.436146 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 6076
I20260808 22:30:53.436306 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:30:53.436420 6189 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:30:53.436641 5167 external_mini_cluster.cc:1428] Setting key 0894e136ff17845c742b9388dce4a5fd
I20260808 22:30:53.443441 6189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58d73d1ede9740f7a11ebd34e01dfb9b: Bootstrap starting.
I20260808 22:30:53.445653 6193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:53.438054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44411 (local address 127.0.0.1:33217)
0808 22:30:53.438515 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:53.438532 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:53.438568 (+ 36us) server_negotiation.cc:407] Connection header received
0808 22:30:53.439189 (+ 621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:53.439204 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:53.439531 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:53.439878 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:53.440197 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:53.441167 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:53.442399 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:53.442863 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:53.442986 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:53.444012 (+ 1026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:53.444017 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:53.444030 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:30:53.444402 (+ 372us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:53.444456 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:30:53.444479 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:53.444625 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:53.444780 (+ 155us) server_negotiation.cc:299] Negotiation successful
0808 22:30:53.444917 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":161,"threads_started":1}
I20260808 22:30:53.445788 6189 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 58d73d1ede9740f7a11ebd34e01dfb9b: Neither blocks nor log segments found. Creating new log.
I20260808 22:30:53.446509 6189 log.cc:824] T 00000000000000000000000000000000 P 58d73d1ede9740f7a11ebd34e01dfb9b: Log is configured to *not* fsync() on all Append() calls
I20260808 22:30:53.448706 5167 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:30:53.448827 6189 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 58d73d1ede9740f7a11ebd34e01dfb9b: No bootstrap required, opened a new log
I20260808 22:30:53.453167 6189 raft_consensus.cc:348] T 00000000000000000000000000000000 P 58d73d1ede9740f7a11ebd34e01dfb9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58d73d1ede9740f7a11ebd34e01dfb9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33217 } }
I20260808 22:30:53.453473 6189 raft_consensus.cc:374] T 00000000000000000000000000000000 P 58d73d1ede9740f7a11ebd34e01dfb9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:30:53.453537 6189 raft_consensus.cc:729] T 00000000000000000000000000000000 P 58d73d1ede9740f7a11ebd34e01dfb9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58d73d1ede9740f7a11ebd34e01dfb9b, State: Initialized, Role: FOLLOWER
I20260808 22:30:53.453979 6189 consensus_queue.cc:260] T 00000000000000000000000000000000 P 58d73d1ede9740f7a11ebd34e01dfb9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58d73d1ede9740f7a11ebd34e01dfb9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33217 } }
I20260808 22:30:53.454152 6189 raft_consensus.cc:388] T 00000000000000000000000000000000 P 58d73d1ede9740f7a11ebd34e01dfb9b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:30:53.454231 6189 raft_consensus.cc:482] T 00000000000000000000000000000000 P 58d73d1ede9740f7a11ebd34e01dfb9b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:30:53.454355 6189 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 58d73d1ede9740f7a11ebd34e01dfb9b [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:30:53.455024 6193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:53.450693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60839 (local address 127.0.0.1:33217)
0808 22:30:53.450802 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:53.450805 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:53.451081 (+ 276us) server_negotiation.cc:407] Connection header received
0808 22:30:53.451215 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:53.451217 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:53.451251 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:53.451315 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:53.451608 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:53.452149 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:53.453255 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:53.453493 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:53.453601 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:53.454585 (+ 984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:53.454591 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:53.454598 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:30:53.454614 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:53.454642 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:30:53.454657 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:53.454756 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:53.454861 (+ 105us) server_negotiation.cc:299] Negotiation successful
0808 22:30:53.454896 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260808 22:30:53.455507 6189 raft_consensus.cc:504] T 00000000000000000000000000000000 P 58d73d1ede9740f7a11ebd34e01dfb9b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58d73d1ede9740f7a11ebd34e01dfb9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33217 } }
I20260808 22:30:53.455893 6189 leader_election.cc:302] T 00000000000000000000000000000000 P 58d73d1ede9740f7a11ebd34e01dfb9b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 58d73d1ede9740f7a11ebd34e01dfb9b; no voters:
I20260808 22:30:53.456259 6189 leader_election.cc:288] T 00000000000000000000000000000000 P 58d73d1ede9740f7a11ebd34e01dfb9b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:30:53.456328 6201 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 58d73d1ede9740f7a11ebd34e01dfb9b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:30:53.456527 6201 raft_consensus.cc:686] T 00000000000000000000000000000000 P 58d73d1ede9740f7a11ebd34e01dfb9b [term 1 LEADER]: Becoming Leader. State: Replica: 58d73d1ede9740f7a11ebd34e01dfb9b, State: Running, Role: LEADER
I20260808 22:30:53.456868 6201 consensus_queue.cc:237] T 00000000000000000000000000000000 P 58d73d1ede9740f7a11ebd34e01dfb9b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58d73d1ede9740f7a11ebd34e01dfb9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33217 } }
I20260808 22:30:53.457545 6189 sys_catalog.cc:564] T 00000000000000000000000000000000 P 58d73d1ede9740f7a11ebd34e01dfb9b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:30:53.458642 6208 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58d73d1ede9740f7a11ebd34e01dfb9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 58d73d1ede9740f7a11ebd34e01dfb9b. Latest consensus state: current_term: 1 leader_uuid: "58d73d1ede9740f7a11ebd34e01dfb9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58d73d1ede9740f7a11ebd34e01dfb9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33217 } } }
I20260808 22:30:53.458671 6203 sys_catalog.cc:455] T 00000000000000000000000000000000 P 58d73d1ede9740f7a11ebd34e01dfb9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "58d73d1ede9740f7a11ebd34e01dfb9b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "58d73d1ede9740f7a11ebd34e01dfb9b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33217 } } }
I20260808 22:30:53.458791 6203 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58d73d1ede9740f7a11ebd34e01dfb9b [sys.catalog]: This master's current role is: LEADER
I20260808 22:30:53.458791 6208 sys_catalog.cc:458] T 00000000000000000000000000000000 P 58d73d1ede9740f7a11ebd34e01dfb9b [sys.catalog]: This master's current role is: LEADER
I20260808 22:30:53.459251 6217 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:30:53.462486 6217 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:30:53.468243 6217 catalog_manager.cc:1347] Generated new cluster ID: c7fc0ccf334f43abb21b11c66d6bc6f6
I20260808 22:30:53.468319 6217 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:30:53.476493 6217 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:30:53.477388 6217 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:30:53.487097 6217 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 58d73d1ede9740f7a11ebd34e01dfb9b: Generated new TSK 0
I20260808 22:30:53.487767 6217 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:30:53.491472 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 6076
W20260808 22:30:53.512208 6046 connection.cc:535] client connection to 127.0.0.1:33217 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33217): Connection reset by peer (error 104)
2026-08-08T22:30:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1633 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:30:53 dist-test-slave-7f01 krb5kdc[6233](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:30:53 dist-test-slave-7f01 krb5kdc[6233](info): set up 2 sockets
Aug 08 22:30:53 dist-test-slave-7f01 krb5kdc[6233](info): commencing operation
krb5kdc: starting...
W20260808 22:30:54.778170 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.251s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:30:54 dist-test-slave-7f01 krb5kdc[6233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228254, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:30:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:30:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 22:30:54.907068 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36423
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45709
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36423
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:30:55.020123 6252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:30:55.020429 6252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:30:55.020505 6252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:30:55.024539 6252 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:30:55.024624 6252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:30:55.024663 6252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:30:55.024720 6252 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:30:55.024761 6252 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:30:55.030007 6252 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45709
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36423
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36423
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:30:55.031076 6252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:30:55.032137 6252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:30:55.040684 6258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:55.040700 6260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:55.040745 6265 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:55.040939 6252 server_base.cc:1034] running on GCE node
I20260808 22:30:55.041622 6252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:30:55.042722 6252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:30:55.043881 6252 hybrid_clock.cc:648] HybridClock initialized: now 1786228255043866 us; error 27 us; skew 500 ppm
Aug 08 22:30:55 dist-test-slave-7f01 krb5kdc[6233](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228255, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:30:55.049685 6252 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 22:30:55.050628 6252 webserver.cc:460] Webserver started at http://127.0.0.1:44039/ using document root <none> and password file <none>
I20260808 22:30:55.051234 6252 fs_manager.cc:359] Metadata directory not provided
I20260808 22:30:55.051328 6252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:30:55.051569 6252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:30:55.053679 6252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "5834f070c9b44dcfb83b69b79b2c8536"
format_stamp: "Formatted at 2026-08-08 22:30:55 on dist-test-slave-7f01"
server_key: "ee39a182f7b2551dd320274efb4a8a18"
server_key_iv: "370514034224cf2dd4883d8bf9bb4a05"
server_key_version: "encryptionkey@0"
I20260808 22:30:55.054251 6252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "5834f070c9b44dcfb83b69b79b2c8536"
format_stamp: "Formatted at 2026-08-08 22:30:55 on dist-test-slave-7f01"
server_key: "ee39a182f7b2551dd320274efb4a8a18"
server_key_iv: "370514034224cf2dd4883d8bf9bb4a05"
server_key_version: "encryptionkey@0"
I20260808 22:30:55.057911 6252 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.002s
I20260808 22:30:55.060376 6282 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:55.061568 6252 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:30:55.061710 6252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "5834f070c9b44dcfb83b69b79b2c8536"
format_stamp: "Formatted at 2026-08-08 22:30:55 on dist-test-slave-7f01"
server_key: "ee39a182f7b2551dd320274efb4a8a18"
server_key_iv: "370514034224cf2dd4883d8bf9bb4a05"
server_key_version: "encryptionkey@0"
I20260808 22:30:55.061821 6252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:30:55.066403 6252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:30:55.069252 6252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:30:55.069445 6252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:30:55.076615 6252 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36423
I20260808 22:30:55.076647 6377 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36423 every 8 connection(s)
I20260808 22:30:55.077507 6252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:30:55.080267 6378 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:30:55.083197 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 6252
I20260808 22:30:55.083359 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:30:55.083664 5167 external_mini_cluster.cc:1428] Setting key c4138ba8dd987f37f90a0d64d160a032
I20260808 22:30:55.086395 6378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5834f070c9b44dcfb83b69b79b2c8536: Bootstrap starting.
I20260808 22:30:55.088893 6381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:55.085111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39293 (local address 127.0.0.1:36423)
0808 22:30:55.085578 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:55.085592 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:55.085628 (+ 36us) server_negotiation.cc:407] Connection header received
0808 22:30:55.086230 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:55.086248 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:55.086575 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:55.086925 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:55.087699 (+ 774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:55.087704 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:55.087715 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:30:55.087726 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:55.087829 (+ 103us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:30:55.087851 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:55.087999 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:55.088068 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 22:30:55.088233 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":145,"threads_started":1}
I20260808 22:30:55.088879 6378 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5834f070c9b44dcfb83b69b79b2c8536: Neither blocks nor log segments found. Creating new log.
I20260808 22:30:55.089815 6378 log.cc:824] T 00000000000000000000000000000000 P 5834f070c9b44dcfb83b69b79b2c8536: Log is configured to *not* fsync() on all Append() calls
I20260808 22:30:55.091717 5167 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:30:55.092051 6378 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5834f070c9b44dcfb83b69b79b2c8536: No bootstrap required, opened a new log
I20260808 22:30:55.095372 6381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:55.093803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35073 (local address 127.0.0.1:36423)
0808 22:30:55.093920 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:55.093923 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:55.094059 (+ 136us) server_negotiation.cc:407] Connection header received
0808 22:30:55.094176 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:55.094178 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:55.094205 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:55.094260 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:55.094957 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:55.094960 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:55.094962 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:30:55.094972 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:55.094996 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:30:55.095006 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:55.095070 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:55.095210 (+ 140us) server_negotiation.cc:299] Negotiation successful
0808 22:30:55.095241 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 22:30:55.096419 6378 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5834f070c9b44dcfb83b69b79b2c8536 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5834f070c9b44dcfb83b69b79b2c8536" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36423 } }
I20260808 22:30:55.096637 6378 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5834f070c9b44dcfb83b69b79b2c8536 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:30:55.096710 6378 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5834f070c9b44dcfb83b69b79b2c8536 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5834f070c9b44dcfb83b69b79b2c8536, State: Initialized, Role: FOLLOWER
I20260808 22:30:55.097117 6378 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5834f070c9b44dcfb83b69b79b2c8536 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5834f070c9b44dcfb83b69b79b2c8536" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36423 } }
I20260808 22:30:55.097258 6378 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5834f070c9b44dcfb83b69b79b2c8536 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:30:55.097326 6378 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5834f070c9b44dcfb83b69b79b2c8536 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:30:55.097433 6378 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5834f070c9b44dcfb83b69b79b2c8536 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:30:55.098309 6378 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5834f070c9b44dcfb83b69b79b2c8536 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5834f070c9b44dcfb83b69b79b2c8536" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36423 } }
I20260808 22:30:55.098657 6378 leader_election.cc:302] T 00000000000000000000000000000000 P 5834f070c9b44dcfb83b69b79b2c8536 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5834f070c9b44dcfb83b69b79b2c8536; no voters:
I20260808 22:30:55.098946 6378 leader_election.cc:288] T 00000000000000000000000000000000 P 5834f070c9b44dcfb83b69b79b2c8536 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:30:55.099033 6392 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5834f070c9b44dcfb83b69b79b2c8536 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:30:55.099321 6392 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5834f070c9b44dcfb83b69b79b2c8536 [term 1 LEADER]: Becoming Leader. State: Replica: 5834f070c9b44dcfb83b69b79b2c8536, State: Running, Role: LEADER
I20260808 22:30:55.099681 6392 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5834f070c9b44dcfb83b69b79b2c8536 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5834f070c9b44dcfb83b69b79b2c8536" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36423 } }
I20260808 22:30:55.100378 6378 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5834f070c9b44dcfb83b69b79b2c8536 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:30:55.101521 6393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5834f070c9b44dcfb83b69b79b2c8536 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5834f070c9b44dcfb83b69b79b2c8536" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5834f070c9b44dcfb83b69b79b2c8536" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36423 } } }
I20260808 22:30:55.101589 6394 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5834f070c9b44dcfb83b69b79b2c8536 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5834f070c9b44dcfb83b69b79b2c8536. Latest consensus state: current_term: 1 leader_uuid: "5834f070c9b44dcfb83b69b79b2c8536" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5834f070c9b44dcfb83b69b79b2c8536" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36423 } } }
I20260808 22:30:55.101658 6393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5834f070c9b44dcfb83b69b79b2c8536 [sys.catalog]: This master's current role is: LEADER
I20260808 22:30:55.101691 6394 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5834f070c9b44dcfb83b69b79b2c8536 [sys.catalog]: This master's current role is: LEADER
I20260808 22:30:55.102010 6405 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:30:55.105160 6405 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:30:55.105507 6420 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260808 22:30:55.110917 6405 catalog_manager.cc:1347] Generated new cluster ID: f5630b7abb98456f8422cf614764e1cc
I20260808 22:30:55.110998 6405 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:30:55.134212 6405 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:30:55.134555 6405 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:30:55.139212 6405 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5834f070c9b44dcfb83b69b79b2c8536: Generated new TSK 0
I20260808 22:30:55.139921 6405 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:30:55.148218 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 6252
2026-08-08T22:30:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1645 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:30:55 dist-test-slave-7f01 krb5kdc[6428](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:30:55 dist-test-slave-7f01 krb5kdc[6428](info): set up 2 sockets
Aug 08 22:30:55 dist-test-slave-7f01 krb5kdc[6428](info): commencing operation
krb5kdc: starting...
W20260808 22:30:56.424088 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.251s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:30:56 dist-test-slave-7f01 krb5kdc[6428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:30:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:30:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 22:30:56.551208 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40891
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40891
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:30:56.663342 6446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:30:56.663652 6446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:30:56.663722 6446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:30:56.667601 6446 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:30:56.667698 6446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:30:56.667735 6446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:30:56.667793 6446 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:30:56.667829 6446 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:30:56.672878 6446 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40891
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:30:56.673882 6446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:30:56.674901 6446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:30:56.683234 6455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:56.683223 6453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:56.683226 6452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:56.683561 6446 server_base.cc:1034] running on GCE node
I20260808 22:30:56.684252 6446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:30:56.685314 6446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:30:56.686513 6446 hybrid_clock.cc:648] HybridClock initialized: now 1786228256686494 us; error 29 us; skew 500 ppm
Aug 08 22:30:56 dist-test-slave-7f01 krb5kdc[6428](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228256, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:30:56.689420 6446 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 22:30:56.690343 6446 webserver.cc:460] Webserver started at http://127.0.0.1:45291/ using document root <none> and password file <none>
I20260808 22:30:56.690912 6446 fs_manager.cc:359] Metadata directory not provided
I20260808 22:30:56.691017 6446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:30:56.691262 6446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:30:56.693368 6446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "192c2fc61b8244a1afbdc1ef19d28988"
format_stamp: "Formatted at 2026-08-08 22:30:56 on dist-test-slave-7f01"
server_key: "561518e95c426631bc80dd0a6cd6c9f1"
server_key_iv: "987b9c2485a97b08d089a051b033991d"
server_key_version: "encryptionkey@0"
I20260808 22:30:56.693959 6446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "192c2fc61b8244a1afbdc1ef19d28988"
format_stamp: "Formatted at 2026-08-08 22:30:56 on dist-test-slave-7f01"
server_key: "561518e95c426631bc80dd0a6cd6c9f1"
server_key_iv: "987b9c2485a97b08d089a051b033991d"
server_key_version: "encryptionkey@0"
I20260808 22:30:56.697781 6446 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 22:30:56.700237 6468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:56.701467 6446 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:30:56.701618 6446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "192c2fc61b8244a1afbdc1ef19d28988"
format_stamp: "Formatted at 2026-08-08 22:30:56 on dist-test-slave-7f01"
server_key: "561518e95c426631bc80dd0a6cd6c9f1"
server_key_iv: "987b9c2485a97b08d089a051b033991d"
server_key_version: "encryptionkey@0"
I20260808 22:30:56.701735 6446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:30:56.719221 6446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:30:56.722626 6446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:30:56.722844 6446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:30:56.730393 6446 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40891
I20260808 22:30:56.730422 6570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40891 every 8 connection(s)
I20260808 22:30:56.731351 6446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:30:56.734206 6571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:30:56.737632 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 6446
I20260808 22:30:56.737787 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:30:56.738080 5167 external_mini_cluster.cc:1428] Setting key 7c3f32c376684c1b96aaf72046fce3db
I20260808 22:30:56.740206 6571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 192c2fc61b8244a1afbdc1ef19d28988: Bootstrap starting.
I20260808 22:30:56.742316 6571 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 192c2fc61b8244a1afbdc1ef19d28988: Neither blocks nor log segments found. Creating new log.
I20260808 22:30:56.743050 6571 log.cc:824] T 00000000000000000000000000000000 P 192c2fc61b8244a1afbdc1ef19d28988: Log is configured to *not* fsync() on all Append() calls
I20260808 22:30:56.744900 6571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 192c2fc61b8244a1afbdc1ef19d28988: No bootstrap required, opened a new log
Aug 08 22:30:56 dist-test-slave-7f01 krb5kdc[6428](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228256, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 22:30:56.748872 6571 raft_consensus.cc:348] T 00000000000000000000000000000000 P 192c2fc61b8244a1afbdc1ef19d28988 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "192c2fc61b8244a1afbdc1ef19d28988" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40891 } }
I20260808 22:30:56.749078 6571 raft_consensus.cc:374] T 00000000000000000000000000000000 P 192c2fc61b8244a1afbdc1ef19d28988 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:30:56.749143 6571 raft_consensus.cc:729] T 00000000000000000000000000000000 P 192c2fc61b8244a1afbdc1ef19d28988 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 192c2fc61b8244a1afbdc1ef19d28988, State: Initialized, Role: FOLLOWER
I20260808 22:30:56.749614 6571 consensus_queue.cc:260] T 00000000000000000000000000000000 P 192c2fc61b8244a1afbdc1ef19d28988 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "192c2fc61b8244a1afbdc1ef19d28988" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40891 } }
I20260808 22:30:56.749804 6571 raft_consensus.cc:388] T 00000000000000000000000000000000 P 192c2fc61b8244a1afbdc1ef19d28988 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:30:56.749879 6571 raft_consensus.cc:482] T 00000000000000000000000000000000 P 192c2fc61b8244a1afbdc1ef19d28988 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:30:56.749987 6571 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 192c2fc61b8244a1afbdc1ef19d28988 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:30:56.751025 6571 raft_consensus.cc:504] T 00000000000000000000000000000000 P 192c2fc61b8244a1afbdc1ef19d28988 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "192c2fc61b8244a1afbdc1ef19d28988" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40891 } }
I20260808 22:30:56.751399 6571 leader_election.cc:302] T 00000000000000000000000000000000 P 192c2fc61b8244a1afbdc1ef19d28988 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 192c2fc61b8244a1afbdc1ef19d28988; no voters:
I20260808 22:30:56.751818 6571 leader_election.cc:288] T 00000000000000000000000000000000 P 192c2fc61b8244a1afbdc1ef19d28988 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:30:56.751897 6576 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 192c2fc61b8244a1afbdc1ef19d28988 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:30:56.752110 6576 raft_consensus.cc:686] T 00000000000000000000000000000000 P 192c2fc61b8244a1afbdc1ef19d28988 [term 1 LEADER]: Becoming Leader. State: Replica: 192c2fc61b8244a1afbdc1ef19d28988, State: Running, Role: LEADER
I20260808 22:30:56.752480 6576 consensus_queue.cc:237] T 00000000000000000000000000000000 P 192c2fc61b8244a1afbdc1ef19d28988 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "192c2fc61b8244a1afbdc1ef19d28988" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40891 } }
I20260808 22:30:56.752857 6574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:56.739507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53181 (local address 127.0.0.1:40891)
0808 22:30:56.739954 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:56.739968 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:56.740004 (+ 36us) server_negotiation.cc:407] Connection header received
0808 22:30:56.740687 (+ 683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:56.740707 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:56.741042 (+ 335us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:56.741374 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:56.742185 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:56.743296 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:56.744209 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:56.744697 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:56.744885 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:56.747614 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:56.747619 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:56.747632 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:30:56.747658 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:56.749577 (+ 1919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:56.750092 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:56.750100 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:56.750106 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:56.750234 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:56.750530 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:56.750532 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:56.750533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:56.750906 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:56.751074 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:56.751402 (+ 328us) server_negotiation.cc:299] Negotiation successful
0808 22:30:56.751837 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":151,"threads_started":1}
I20260808 22:30:56.753088 6571 sys_catalog.cc:564] T 00000000000000000000000000000000 P 192c2fc61b8244a1afbdc1ef19d28988 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:30:56.753708 6576 sys_catalog.cc:455] T 00000000000000000000000000000000 P 192c2fc61b8244a1afbdc1ef19d28988 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 192c2fc61b8244a1afbdc1ef19d28988. Latest consensus state: current_term: 1 leader_uuid: "192c2fc61b8244a1afbdc1ef19d28988" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "192c2fc61b8244a1afbdc1ef19d28988" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40891 } } }
I20260808 22:30:56.754436 6576 sys_catalog.cc:458] T 00000000000000000000000000000000 P 192c2fc61b8244a1afbdc1ef19d28988 [sys.catalog]: This master's current role is: LEADER
I20260808 22:30:56.754324 6577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 192c2fc61b8244a1afbdc1ef19d28988 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "192c2fc61b8244a1afbdc1ef19d28988" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "192c2fc61b8244a1afbdc1ef19d28988" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40891 } } }
I20260808 22:30:56.754742 6577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 192c2fc61b8244a1afbdc1ef19d28988 [sys.catalog]: This master's current role is: LEADER
I20260808 22:30:56.756204 5167 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260808 22:30:56.758383 6599 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 192c2fc61b8244a1afbdc1ef19d28988: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:30:56.758483 6599 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:30:56.758570 6590 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:30:56.759533 6590 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:30:56.764748 6590 catalog_manager.cc:1347] Generated new cluster ID: 21a9661d9ca04be0b937aa8b9fd5309a
I20260808 22:30:56.764808 6590 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:30:56.768623 6574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:56.758969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55763 (local address 127.0.0.1:40891)
0808 22:30:56.759079 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:56.759084 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:56.759333 (+ 249us) server_negotiation.cc:407] Connection header received
0808 22:30:56.759435 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:56.759438 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:56.759495 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:56.759584 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:56.760433 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:56.761245 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:56.762487 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:56.762924 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:56.763005 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:56.764881 (+ 1876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:56.764885 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:56.764888 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:30:56.764914 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:56.766621 (+ 1707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:56.767157 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:56.767160 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:56.767162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:56.767251 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:30:56.767587 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:30:56.767591 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:30:56.767593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:30:56.767854 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:56.767964 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:56.768338 (+ 374us) server_negotiation.cc:299] Negotiation successful
0808 22:30:56.768468 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 22:30:56.773186 6590 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:30:56.774037 6590 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:30:56.778054 6590 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 192c2fc61b8244a1afbdc1ef19d28988: Generated new TSK 0
I20260808 22:30:56.778653 6590 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:30:56.782325 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 6446
2026-08-08T22:30:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1640 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:30:56 dist-test-slave-7f01 krb5kdc[6629](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:30:56 dist-test-slave-7f01 krb5kdc[6629](info): set up 2 sockets
Aug 08 22:30:56 dist-test-slave-7f01 krb5kdc[6629](info): commencing operation
krb5kdc: starting...
W20260808 22:30:58.049213 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:30:58 dist-test-slave-7f01 krb5kdc[6629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228258, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:30:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:30:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 22:30:58.172839 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39537
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39537
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:30:58.283061 6646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:30:58.283304 6646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:30:58.283344 6646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:30:58.287178 6646 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:30:58.287228 6646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:30:58.287248 6646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:30:58.287264 6646 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:30:58.287279 6646 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:30:58.292201 6646 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39537
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39537
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:30:58.293079 6646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:30:58.294013 6646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:30:58.302346 6657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:58.302402 6655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:58.302366 6661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:58.302704 6646 server_base.cc:1034] running on GCE node
I20260808 22:30:58.303303 6646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:30:58.304421 6646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:30:58.305624 6646 hybrid_clock.cc:648] HybridClock initialized: now 1786228258305605 us; error 32 us; skew 500 ppm
Aug 08 22:30:58 dist-test-slave-7f01 krb5kdc[6629](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228258, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:30:58.308501 6646 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 22:30:58.309412 6646 webserver.cc:460] Webserver started at http://127.0.0.1:44367/ using document root <none> and password file <none>
I20260808 22:30:58.309927 6646 fs_manager.cc:359] Metadata directory not provided
I20260808 22:30:58.309986 6646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:30:58.310153 6646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:30:58.312199 6646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "a12fb9dee3594b9a95501ccc4225ed3a"
format_stamp: "Formatted at 2026-08-08 22:30:58 on dist-test-slave-7f01"
server_key: "f1eb94f78df107ce065339d9c257d441"
server_key_iv: "25827f84bc17b257e66174446a9bede3"
server_key_version: "encryptionkey@0"
I20260808 22:30:58.312723 6646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "a12fb9dee3594b9a95501ccc4225ed3a"
format_stamp: "Formatted at 2026-08-08 22:30:58 on dist-test-slave-7f01"
server_key: "f1eb94f78df107ce065339d9c257d441"
server_key_iv: "25827f84bc17b257e66174446a9bede3"
server_key_version: "encryptionkey@0"
I20260808 22:30:58.316293 6646 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260808 22:30:58.318645 6681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:58.319890 6646 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:30:58.319993 6646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "a12fb9dee3594b9a95501ccc4225ed3a"
format_stamp: "Formatted at 2026-08-08 22:30:58 on dist-test-slave-7f01"
server_key: "f1eb94f78df107ce065339d9c257d441"
server_key_iv: "25827f84bc17b257e66174446a9bede3"
server_key_version: "encryptionkey@0"
I20260808 22:30:58.320087 6646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:30:58.336341 6646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:30:58.339093 6646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:30:58.339241 6646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:30:58.346242 6646 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39537
I20260808 22:30:58.346328 6776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39537 every 8 connection(s)
I20260808 22:30:58.347182 6646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:30:58.348395 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 6646
I20260808 22:30:58.348551 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:30:58.348856 5167 external_mini_cluster.cc:1428] Setting key dbc1bedda7db2de42c7913f3e87dfe6b
I20260808 22:30:58.350270 6778 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:30:58.357232 6778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a12fb9dee3594b9a95501ccc4225ed3a: Bootstrap starting.
I20260808 22:30:58.358693 6782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:58.350249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36109 (local address 127.0.0.1:39537)
0808 22:30:58.350708 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:58.350726 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:58.350768 (+ 42us) server_negotiation.cc:407] Connection header received
0808 22:30:58.351599 (+ 831us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:58.351620 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:58.352079 (+ 459us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:58.352426 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:58.352817 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:58.354311 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:58.355162 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:58.355891 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:58.356062 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:58.356808 (+ 746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:58.356814 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:58.356831 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:30:58.357310 (+ 479us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:58.357379 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:30:58.357409 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:58.357508 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:58.357615 (+ 107us) server_negotiation.cc:299] Negotiation successful
0808 22:30:58.357808 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":163,"threads_started":1}
I20260808 22:30:58.359701 6778 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a12fb9dee3594b9a95501ccc4225ed3a: Neither blocks nor log segments found. Creating new log.
I20260808 22:30:58.360517 6778 log.cc:824] T 00000000000000000000000000000000 P a12fb9dee3594b9a95501ccc4225ed3a: Log is configured to *not* fsync() on all Append() calls
I20260808 22:30:58.361438 5167 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:30:58.362473 6778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a12fb9dee3594b9a95501ccc4225ed3a: No bootstrap required, opened a new log
I20260808 22:30:58.366746 6778 raft_consensus.cc:348] T 00000000000000000000000000000000 P a12fb9dee3594b9a95501ccc4225ed3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a12fb9dee3594b9a95501ccc4225ed3a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39537 } }
I20260808 22:30:58.367064 6778 raft_consensus.cc:374] T 00000000000000000000000000000000 P a12fb9dee3594b9a95501ccc4225ed3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:30:58.367133 6778 raft_consensus.cc:729] T 00000000000000000000000000000000 P a12fb9dee3594b9a95501ccc4225ed3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a12fb9dee3594b9a95501ccc4225ed3a, State: Initialized, Role: FOLLOWER
I20260808 22:30:58.367381 6782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:30:58.363411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43805 (local address 127.0.0.1:39537)
0808 22:30:58.363538 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 22:30:58.363543 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:30:58.363722 (+ 179us) server_negotiation.cc:407] Connection header received
0808 22:30:58.363851 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:30:58.363853 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:30:58.363887 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:30:58.363949 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:30:58.364246 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:58.364723 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:58.365884 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:30:58.366109 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:30:58.366182 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:30:58.366942 (+ 760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:30:58.366945 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:30:58.366947 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:30:58.366958 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:30:58.366979 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:30:58.366989 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:30:58.367079 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 22:30:58.367198 (+ 119us) server_negotiation.cc:299] Negotiation successful
0808 22:30:58.367242 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 22:30:58.367621 6778 consensus_queue.cc:260] T 00000000000000000000000000000000 P a12fb9dee3594b9a95501ccc4225ed3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a12fb9dee3594b9a95501ccc4225ed3a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39537 } }
I20260808 22:30:58.367810 6778 raft_consensus.cc:388] T 00000000000000000000000000000000 P a12fb9dee3594b9a95501ccc4225ed3a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:30:58.367883 6778 raft_consensus.cc:482] T 00000000000000000000000000000000 P a12fb9dee3594b9a95501ccc4225ed3a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:30:58.367985 6778 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a12fb9dee3594b9a95501ccc4225ed3a [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:30:58.368932 6778 raft_consensus.cc:504] T 00000000000000000000000000000000 P a12fb9dee3594b9a95501ccc4225ed3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a12fb9dee3594b9a95501ccc4225ed3a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39537 } }
I20260808 22:30:58.369273 6778 leader_election.cc:302] T 00000000000000000000000000000000 P a12fb9dee3594b9a95501ccc4225ed3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a12fb9dee3594b9a95501ccc4225ed3a; no voters:
I20260808 22:30:58.369618 6778 leader_election.cc:288] T 00000000000000000000000000000000 P a12fb9dee3594b9a95501ccc4225ed3a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:30:58.369700 6795 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a12fb9dee3594b9a95501ccc4225ed3a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:30:58.369982 6795 raft_consensus.cc:686] T 00000000000000000000000000000000 P a12fb9dee3594b9a95501ccc4225ed3a [term 1 LEADER]: Becoming Leader. State: Replica: a12fb9dee3594b9a95501ccc4225ed3a, State: Running, Role: LEADER
I20260808 22:30:58.370311 6795 consensus_queue.cc:237] T 00000000000000000000000000000000 P a12fb9dee3594b9a95501ccc4225ed3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a12fb9dee3594b9a95501ccc4225ed3a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39537 } }
I20260808 22:30:58.370812 6778 sys_catalog.cc:564] T 00000000000000000000000000000000 P a12fb9dee3594b9a95501ccc4225ed3a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:30:58.372089 6797 sys_catalog.cc:455] T 00000000000000000000000000000000 P a12fb9dee3594b9a95501ccc4225ed3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader a12fb9dee3594b9a95501ccc4225ed3a. Latest consensus state: current_term: 1 leader_uuid: "a12fb9dee3594b9a95501ccc4225ed3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a12fb9dee3594b9a95501ccc4225ed3a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39537 } } }
I20260808 22:30:58.372219 6797 sys_catalog.cc:458] T 00000000000000000000000000000000 P a12fb9dee3594b9a95501ccc4225ed3a [sys.catalog]: This master's current role is: LEADER
I20260808 22:30:58.372381 6796 sys_catalog.cc:455] T 00000000000000000000000000000000 P a12fb9dee3594b9a95501ccc4225ed3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a12fb9dee3594b9a95501ccc4225ed3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a12fb9dee3594b9a95501ccc4225ed3a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39537 } } }
I20260808 22:30:58.372545 6796 sys_catalog.cc:458] T 00000000000000000000000000000000 P a12fb9dee3594b9a95501ccc4225ed3a [sys.catalog]: This master's current role is: LEADER
I20260808 22:30:58.373445 6807 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:30:58.376545 6807 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:30:58.381521 6807 catalog_manager.cc:1347] Generated new cluster ID: f484428aa75c47bf8e2fb9a50c03e6b7
I20260808 22:30:58.381584 6807 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:30:58.394398 6807 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:30:58.395298 6807 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:30:58.402065 6807 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a12fb9dee3594b9a95501ccc4225ed3a: Generated new TSK 0
I20260808 22:30:58.402710 6807 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:30:58.415544 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 6646
2026-08-08T22:30:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1627 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:30:58 dist-test-slave-7f01 krb5kdc[6833](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:30:58 dist-test-slave-7f01 krb5kdc[6833](info): set up 2 sockets
Aug 08 22:30:58 dist-test-slave-7f01 krb5kdc[6833](info): commencing operation
krb5kdc: starting...
W20260808 22:30:59.711165 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.270s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:30:59 dist-test-slave-7f01 krb5kdc[6833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228259, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:30:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:30:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 22:30:59.839186 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43755
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43755
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:30:59.957643 6860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:30:59.957901 6860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:30:59.957978 6860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:30:59.961892 6860 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:30:59.961949 6860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:30:59.961969 6860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:30:59.961985 6860 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:30:59.962001 6860 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:30:59.967274 6860 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43755
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43755
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:30:59.968317 6860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:30:59.969332 6860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:30:59.977885 6868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:59.977994 6860 server_base.cc:1034] running on GCE node
W20260808 22:30:59.977891 6873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:30:59.977887 6867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:30:59.978770 6860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:30:59.979912 6860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:30:59.981076 6860 hybrid_clock.cc:648] HybridClock initialized: now 1786228259981057 us; error 28 us; skew 500 ppm
Aug 08 22:30:59 dist-test-slave-7f01 krb5kdc[6833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228259, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:30:59.983865 6860 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 22:30:59.984735 6860 webserver.cc:460] Webserver started at http://127.0.0.1:33743/ using document root <none> and password file <none>
I20260808 22:30:59.985313 6860 fs_manager.cc:359] Metadata directory not provided
I20260808 22:30:59.985419 6860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:30:59.985661 6860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:30:59.987807 6860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "01549fc17f92435b93cbd1fa75964be3"
format_stamp: "Formatted at 2026-08-08 22:30:59 on dist-test-slave-7f01"
server_key: "52846e7fc5aa6328d5555265dcb66e3f"
server_key_iv: "136b23b29bd6337115c5a70c257445e0"
server_key_version: "encryptionkey@0"
I20260808 22:30:59.988426 6860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "01549fc17f92435b93cbd1fa75964be3"
format_stamp: "Formatted at 2026-08-08 22:30:59 on dist-test-slave-7f01"
server_key: "52846e7fc5aa6328d5555265dcb66e3f"
server_key_iv: "136b23b29bd6337115c5a70c257445e0"
server_key_version: "encryptionkey@0"
I20260808 22:30:59.992193 6860 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:30:59.994643 6884 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:30:59.995932 6860 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:30:59.996073 6860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "01549fc17f92435b93cbd1fa75964be3"
format_stamp: "Formatted at 2026-08-08 22:30:59 on dist-test-slave-7f01"
server_key: "52846e7fc5aa6328d5555265dcb66e3f"
server_key_iv: "136b23b29bd6337115c5a70c257445e0"
server_key_version: "encryptionkey@0"
I20260808 22:30:59.996189 6860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:00.000804 6860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:00.003671 6860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:00.003896 6860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:00.011246 6860 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43755
I20260808 22:31:00.011272 6975 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43755 every 8 connection(s)
I20260808 22:31:00.012190 6860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:31:00.015094 6976 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:00.015522 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 6860
I20260808 22:31:00.015681 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:31:00.016036 5167 external_mini_cluster.cc:1428] Setting key 78ae4455ef804902ff7f784ff69c4415
I20260808 22:31:00.021216 6980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:00.017357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36643 (local address 127.0.0.1:43755)
0808 22:31:00.017776 (+ 419us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:00.017792 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:00.017826 (+ 34us) server_negotiation.cc:407] Connection header received
0808 22:31:00.018524 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:00.018542 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:00.018816 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:00.019096 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:00.020022 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:00.020027 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:00.020039 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:31:00.020050 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:00.020105 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:31:00.020125 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:00.020230 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:00.020351 (+ 121us) server_negotiation.cc:299] Negotiation successful
0808 22:31:00.020524 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":131,"threads_started":1}
I20260808 22:31:00.022236 6976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01549fc17f92435b93cbd1fa75964be3: Bootstrap starting.
I20260808 22:31:00.024163 5167 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:31:00.024953 6976 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01549fc17f92435b93cbd1fa75964be3: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:00.025888 6976 log.cc:824] T 00000000000000000000000000000000 P 01549fc17f92435b93cbd1fa75964be3: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:00.027915 6976 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01549fc17f92435b93cbd1fa75964be3: No bootstrap required, opened a new log
I20260808 22:31:00.029383 6980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:00.026539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50405 (local address 127.0.0.1:43755)
0808 22:31:00.026676 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:00.026681 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:00.028059 (+ 1378us) server_negotiation.cc:407] Connection header received
0808 22:31:00.028147 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:00.028149 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:00.028178 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:00.028244 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:00.028954 (+ 710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:00.028956 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:00.028958 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:31:00.028968 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:00.028999 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:31:00.029009 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:00.029078 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:00.029204 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 22:31:00.029235 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 22:31:00.032127 6976 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01549fc17f92435b93cbd1fa75964be3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01549fc17f92435b93cbd1fa75964be3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43755 } }
I20260808 22:31:00.032352 6976 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01549fc17f92435b93cbd1fa75964be3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:00.032434 6976 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01549fc17f92435b93cbd1fa75964be3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01549fc17f92435b93cbd1fa75964be3, State: Initialized, Role: FOLLOWER
I20260808 22:31:00.032920 6976 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01549fc17f92435b93cbd1fa75964be3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01549fc17f92435b93cbd1fa75964be3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43755 } }
I20260808 22:31:00.033087 6976 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01549fc17f92435b93cbd1fa75964be3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:31:00.033172 6976 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01549fc17f92435b93cbd1fa75964be3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:31:00.033279 6976 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01549fc17f92435b93cbd1fa75964be3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:00.034178 6976 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01549fc17f92435b93cbd1fa75964be3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01549fc17f92435b93cbd1fa75964be3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43755 } }
I20260808 22:31:00.034528 6976 leader_election.cc:302] T 00000000000000000000000000000000 P 01549fc17f92435b93cbd1fa75964be3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01549fc17f92435b93cbd1fa75964be3; no voters:
I20260808 22:31:00.034842 6976 leader_election.cc:288] T 00000000000000000000000000000000 P 01549fc17f92435b93cbd1fa75964be3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:31:00.034929 6992 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01549fc17f92435b93cbd1fa75964be3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:00.035202 6992 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01549fc17f92435b93cbd1fa75964be3 [term 1 LEADER]: Becoming Leader. State: Replica: 01549fc17f92435b93cbd1fa75964be3, State: Running, Role: LEADER
I20260808 22:31:00.035575 6992 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01549fc17f92435b93cbd1fa75964be3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01549fc17f92435b93cbd1fa75964be3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43755 } }
I20260808 22:31:00.036162 6976 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01549fc17f92435b93cbd1fa75964be3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:31:00.037530 6995 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01549fc17f92435b93cbd1fa75964be3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01549fc17f92435b93cbd1fa75964be3. Latest consensus state: current_term: 1 leader_uuid: "01549fc17f92435b93cbd1fa75964be3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01549fc17f92435b93cbd1fa75964be3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43755 } } }
I20260808 22:31:00.037556 6994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01549fc17f92435b93cbd1fa75964be3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01549fc17f92435b93cbd1fa75964be3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01549fc17f92435b93cbd1fa75964be3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43755 } } }
I20260808 22:31:00.037658 6994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01549fc17f92435b93cbd1fa75964be3 [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:00.037657 6995 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01549fc17f92435b93cbd1fa75964be3 [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:00.038782 7011 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:31:00.041445 7011 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:31:00.046329 7011 catalog_manager.cc:1347] Generated new cluster ID: 5f535822e5b942e9830a789b391f4758
I20260808 22:31:00.046392 7011 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:31:00.067425 7011 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:31:00.067775 7011 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:31:00.071786 7011 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01549fc17f92435b93cbd1fa75964be3: Generated new TSK 0
I20260808 22:31:00.072402 7011 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:31:00.084182 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 6860
2026-08-08T22:31:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1669 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:31:00 dist-test-slave-7f01 krb5kdc[7040](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:31:00 dist-test-slave-7f01 krb5kdc[7040](info): set up 2 sockets
Aug 08 22:31:00 dist-test-slave-7f01 krb5kdc[7040](info): commencing operation
krb5kdc: starting...
W20260808 22:31:01.356079 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.245s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:31:01 dist-test-slave-7f01 krb5kdc[7040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:31:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:31:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 22:31:01.482904 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33005
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43115
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33005
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:01.595057 7070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:01.595371 7070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:01.595451 7070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:01.599506 7070 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:31:01.599591 7070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:01.599632 7070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:01.599689 7070 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:31:01.599723 7070 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:31:01.605191 7070 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43115
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33005
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:01.606312 7070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:01.607427 7070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:01.616559 7081 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:01.616593 7084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:01.616817 7070 server_base.cc:1034] running on GCE node
W20260808 22:31:01.616595 7080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:01.617560 7070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:01.618721 7070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:01.619927 7070 hybrid_clock.cc:648] HybridClock initialized: now 1786228261619889 us; error 46 us; skew 500 ppm
Aug 08 22:31:01 dist-test-slave-7f01 krb5kdc[7040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228261, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:01.622821 7070 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 22:31:01.623805 7070 webserver.cc:460] Webserver started at http://127.0.0.1:46453/ using document root <none> and password file <none>
I20260808 22:31:01.624379 7070 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:01.624440 7070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:01.624614 7070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:01.626681 7070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "7bedae569d9d44c5b5a7fa39156cd6d3"
format_stamp: "Formatted at 2026-08-08 22:31:01 on dist-test-slave-7f01"
server_key: "cac25126baf0e85eb5f3857e14e66116"
server_key_iv: "92febc0e571dcdc54677acd09410e96a"
server_key_version: "encryptionkey@0"
I20260808 22:31:01.627250 7070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "7bedae569d9d44c5b5a7fa39156cd6d3"
format_stamp: "Formatted at 2026-08-08 22:31:01 on dist-test-slave-7f01"
server_key: "cac25126baf0e85eb5f3857e14e66116"
server_key_iv: "92febc0e571dcdc54677acd09410e96a"
server_key_version: "encryptionkey@0"
I20260808 22:31:01.631120 7070 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260808 22:31:01.633738 7095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:01.634996 7070 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:31:01.635107 7070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "7bedae569d9d44c5b5a7fa39156cd6d3"
format_stamp: "Formatted at 2026-08-08 22:31:01 on dist-test-slave-7f01"
server_key: "cac25126baf0e85eb5f3857e14e66116"
server_key_iv: "92febc0e571dcdc54677acd09410e96a"
server_key_version: "encryptionkey@0"
I20260808 22:31:01.635200 7070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:01.666143 7070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:01.669386 7070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:01.669551 7070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:01.676659 7070 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33005
I20260808 22:31:01.676690 7186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33005 every 8 connection(s)
I20260808 22:31:01.677639 7070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:31:01.679591 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 7070
I20260808 22:31:01.679796 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:31:01.680133 5167 external_mini_cluster.cc:1428] Setting key e0e87b0c90dac2749fd9af543ecc4b3c
I20260808 22:31:01.681353 7187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:01.687150 7187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bedae569d9d44c5b5a7fa39156cd6d3: Bootstrap starting.
Aug 08 22:31:01 dist-test-slave-7f01 krb5kdc[7040](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228261, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 22:31:01.689787 7187 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7bedae569d9d44c5b5a7fa39156cd6d3: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:01.690740 7187 log.cc:824] T 00000000000000000000000000000000 P 7bedae569d9d44c5b5a7fa39156cd6d3: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:01.692847 7187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bedae569d9d44c5b5a7fa39156cd6d3: No bootstrap required, opened a new log
I20260808 22:31:01.694777 7192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:01.681741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.49:47871 (local address 127.0.0.1:33005)
0808 22:31:01.682192 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:01.682208 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:01.682245 (+ 37us) server_negotiation.cc:407] Connection header received
0808 22:31:01.682887 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:01.682903 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:01.683248 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:01.683583 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:01.684436 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:01.685432 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:01.686287 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:01.686755 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:01.686938 (+ 183us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:01.689761 (+ 2823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:01.689766 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:01.689780 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:01.689807 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:01.691835 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:01.692347 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:01.692353 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:01.692366 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:01.692476 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:01.692830 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:01.692834 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:01.692836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:01.693303 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:01.693534 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:01.693858 (+ 324us) server_negotiation.cc:299] Negotiation successful
0808 22:31:01.694086 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":144,"threads_started":1}
I20260808 22:31:01.696822 7187 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7bedae569d9d44c5b5a7fa39156cd6d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bedae569d9d44c5b5a7fa39156cd6d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33005 } }
I20260808 22:31:01.697084 7187 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7bedae569d9d44c5b5a7fa39156cd6d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:01.697163 7187 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7bedae569d9d44c5b5a7fa39156cd6d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bedae569d9d44c5b5a7fa39156cd6d3, State: Initialized, Role: FOLLOWER
I20260808 22:31:01.697679 7187 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7bedae569d9d44c5b5a7fa39156cd6d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bedae569d9d44c5b5a7fa39156cd6d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33005 } }
I20260808 22:31:01.697872 7187 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7bedae569d9d44c5b5a7fa39156cd6d3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:31:01.697949 7187 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7bedae569d9d44c5b5a7fa39156cd6d3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:31:01.698053 5167 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:31:01.698057 7187 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7bedae569d9d44c5b5a7fa39156cd6d3 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:01.699000 7187 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7bedae569d9d44c5b5a7fa39156cd6d3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bedae569d9d44c5b5a7fa39156cd6d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33005 } }
I20260808 22:31:01.699388 7187 leader_election.cc:302] T 00000000000000000000000000000000 P 7bedae569d9d44c5b5a7fa39156cd6d3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7bedae569d9d44c5b5a7fa39156cd6d3; no voters:
I20260808 22:31:01.699786 7187 leader_election.cc:288] T 00000000000000000000000000000000 P 7bedae569d9d44c5b5a7fa39156cd6d3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:31:01.699855 7198 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7bedae569d9d44c5b5a7fa39156cd6d3 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:01.700050 7198 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7bedae569d9d44c5b5a7fa39156cd6d3 [term 1 LEADER]: Becoming Leader. State: Replica: 7bedae569d9d44c5b5a7fa39156cd6d3, State: Running, Role: LEADER
I20260808 22:31:01.700358 7198 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7bedae569d9d44c5b5a7fa39156cd6d3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bedae569d9d44c5b5a7fa39156cd6d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33005 } }
I20260808 22:31:01.701092 7187 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7bedae569d9d44c5b5a7fa39156cd6d3 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:31:01.701596 7198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bedae569d9d44c5b5a7fa39156cd6d3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7bedae569d9d44c5b5a7fa39156cd6d3. Latest consensus state: current_term: 1 leader_uuid: "7bedae569d9d44c5b5a7fa39156cd6d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bedae569d9d44c5b5a7fa39156cd6d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33005 } } }
I20260808 22:31:01.701714 7198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bedae569d9d44c5b5a7fa39156cd6d3 [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:01.701923 7198 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bedae569d9d44c5b5a7fa39156cd6d3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7bedae569d9d44c5b5a7fa39156cd6d3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bedae569d9d44c5b5a7fa39156cd6d3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33005 } } }
I20260808 22:31:01.702030 7198 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bedae569d9d44c5b5a7fa39156cd6d3 [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:01.702343 7211 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:31:01.705717 7211 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:31:01.712069 7211 catalog_manager.cc:1347] Generated new cluster ID: 87742a91ea1347ff8cf767b258b065e0
I20260808 22:31:01.712138 7211 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:31:01.713173 7192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:01.700120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.49:33461 (local address 127.0.0.1:33005)
0808 22:31:01.700304 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:01.700308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:01.703505 (+ 3197us) server_negotiation.cc:407] Connection header received
0808 22:31:01.703641 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:01.703644 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:01.703697 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:01.703806 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:01.704485 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:01.705293 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:01.706626 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:01.706886 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:01.707000 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:01.708682 (+ 1682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:01.708688 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:01.708691 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:01.708718 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:01.710983 (+ 2265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:01.711555 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:01.711559 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:01.711560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:01.711853 (+ 293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:01.712278 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:01.712281 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:01.712283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:01.712477 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:01.712639 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:01.712913 (+ 274us) server_negotiation.cc:299] Negotiation successful
0808 22:31:01.713011 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260808 22:31:01.730338 7211 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:31:01.731312 7211 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:31:01.741705 7211 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7bedae569d9d44c5b5a7fa39156cd6d3: Generated new TSK 0
I20260808 22:31:01.742408 7211 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:31:01.750483 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 7070
W20260808 22:31:01.767321 7037 connection.cc:535] client connection to 127.0.0.1:33005 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33005): Connection reset by peer (error 104)
2026-08-08T22:31:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1671 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:31:01 dist-test-slave-7f01 krb5kdc[7238](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:31:01 dist-test-slave-7f01 krb5kdc[7238](info): set up 2 sockets
Aug 08 22:31:01 dist-test-slave-7f01 krb5kdc[7238](info): commencing operation
krb5kdc: starting...
W20260808 22:31:03.042932 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.261s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:31:03 dist-test-slave-7f01 krb5kdc[7238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228263, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:31:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:31:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 22:31:03.171583 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42541
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42541
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:03.282179 7266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:03.282449 7266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:03.282492 7266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:03.286489 7266 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:31:03.286561 7266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:03.286580 7266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:03.286597 7266 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:31:03.286612 7266 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:31:03.292004 7266 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42541
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:03.293103 7266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:03.294209 7266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:03.303347 7281 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:03.303305 7278 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:03.303292 7279 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:03.303452 7266 server_base.cc:1034] running on GCE node
I20260808 22:31:03.304229 7266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:03.305382 7266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:03.306577 7266 hybrid_clock.cc:648] HybridClock initialized: now 1786228263306552 us; error 48 us; skew 500 ppm
Aug 08 22:31:03 dist-test-slave-7f01 krb5kdc[7238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228263, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:03.309559 7266 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 22:31:03.310436 7266 webserver.cc:460] Webserver started at http://127.0.0.1:38825/ using document root <none> and password file <none>
I20260808 22:31:03.311023 7266 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:03.311118 7266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:03.311354 7266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:03.313504 7266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "1cde7204582a4d949ce0496273f863e4"
format_stamp: "Formatted at 2026-08-08 22:31:03 on dist-test-slave-7f01"
server_key: "7f699e91b9881e29783a2062e0476f32"
server_key_iv: "523c87393232f17774f6607f47088940"
server_key_version: "encryptionkey@0"
I20260808 22:31:03.314101 7266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "1cde7204582a4d949ce0496273f863e4"
format_stamp: "Formatted at 2026-08-08 22:31:03 on dist-test-slave-7f01"
server_key: "7f699e91b9881e29783a2062e0476f32"
server_key_iv: "523c87393232f17774f6607f47088940"
server_key_version: "encryptionkey@0"
I20260808 22:31:03.317865 7266 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.005s
I20260808 22:31:03.320289 7295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:03.321557 7266 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 22:31:03.321700 7266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "1cde7204582a4d949ce0496273f863e4"
format_stamp: "Formatted at 2026-08-08 22:31:03 on dist-test-slave-7f01"
server_key: "7f699e91b9881e29783a2062e0476f32"
server_key_iv: "523c87393232f17774f6607f47088940"
server_key_version: "encryptionkey@0"
I20260808 22:31:03.321815 7266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:03.332762 7266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:03.335386 7266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:03.335579 7266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:03.342859 7266 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42541
I20260808 22:31:03.342870 7394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42541 every 8 connection(s)
I20260808 22:31:03.343814 7266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:31:03.346494 7395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:03.347886 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 7266
I20260808 22:31:03.348040 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:31:03.348340 5167 external_mini_cluster.cc:1428] Setting key 5543b4bb93a2340352100a48ca6d4518
I20260808 22:31:03.352663 7395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cde7204582a4d949ce0496273f863e4: Bootstrap starting.
I20260808 22:31:03.354853 7395 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1cde7204582a4d949ce0496273f863e4: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:03.355619 7395 log.cc:824] T 00000000000000000000000000000000 P 1cde7204582a4d949ce0496273f863e4: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:03.357091 7404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:03.349700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.49:56009 (local address 127.0.0.1:42541)
0808 22:31:03.350134 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:03.350149 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:03.350182 (+ 33us) server_negotiation.cc:407] Connection header received
0808 22:31:03.350794 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:03.350811 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:03.351133 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:03.351449 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:03.351904 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:03.352889 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:03.353770 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:03.354231 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:03.354400 (+ 169us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:03.355450 (+ 1050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:03.355455 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:03.355467 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:31:03.355894 (+ 427us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:03.355951 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:31:03.355980 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:03.356121 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:03.356305 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 22:31:03.356433 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":141,"threads_started":1}
I20260808 22:31:03.357559 7395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1cde7204582a4d949ce0496273f863e4: No bootstrap required, opened a new log
I20260808 22:31:03.360013 5167 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:31:03.361831 7395 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1cde7204582a4d949ce0496273f863e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cde7204582a4d949ce0496273f863e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42541 } }
I20260808 22:31:03.362118 7395 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1cde7204582a4d949ce0496273f863e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:03.362185 7395 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1cde7204582a4d949ce0496273f863e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cde7204582a4d949ce0496273f863e4, State: Initialized, Role: FOLLOWER
I20260808 22:31:03.362673 7395 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1cde7204582a4d949ce0496273f863e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cde7204582a4d949ce0496273f863e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42541 } }
I20260808 22:31:03.362845 7395 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1cde7204582a4d949ce0496273f863e4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:31:03.362931 7395 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1cde7204582a4d949ce0496273f863e4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:31:03.363039 7395 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1cde7204582a4d949ce0496273f863e4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:03.364094 7395 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1cde7204582a4d949ce0496273f863e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cde7204582a4d949ce0496273f863e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42541 } }
I20260808 22:31:03.364441 7395 leader_election.cc:302] T 00000000000000000000000000000000 P 1cde7204582a4d949ce0496273f863e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1cde7204582a4d949ce0496273f863e4; no voters:
I20260808 22:31:03.364779 7395 leader_election.cc:288] T 00000000000000000000000000000000 P 1cde7204582a4d949ce0496273f863e4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:31:03.364872 7421 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1cde7204582a4d949ce0496273f863e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:03.365168 7421 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1cde7204582a4d949ce0496273f863e4 [term 1 LEADER]: Becoming Leader. State: Replica: 1cde7204582a4d949ce0496273f863e4, State: Running, Role: LEADER
I20260808 22:31:03.365559 7421 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1cde7204582a4d949ce0496273f863e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cde7204582a4d949ce0496273f863e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42541 } }
I20260808 22:31:03.366178 7395 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1cde7204582a4d949ce0496273f863e4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:31:03.366880 7404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:03.361931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.49:35877 (local address 127.0.0.1:42541)
0808 22:31:03.362041 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:03.362044 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:03.362282 (+ 238us) server_negotiation.cc:407] Connection header received
0808 22:31:03.362461 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:03.362467 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:03.362515 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:03.362590 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:03.362949 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:03.363535 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:03.364731 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:03.365179 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:03.365291 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:03.366230 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:03.366237 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:03.366241 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:31:03.366271 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:03.366300 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:31:03.366314 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:03.366406 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:03.366707 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 22:31:03.366744 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260808 22:31:03.366843 7421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cde7204582a4d949ce0496273f863e4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1cde7204582a4d949ce0496273f863e4. Latest consensus state: current_term: 1 leader_uuid: "1cde7204582a4d949ce0496273f863e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cde7204582a4d949ce0496273f863e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42541 } } }
I20260808 22:31:03.366950 7421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cde7204582a4d949ce0496273f863e4 [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:03.367302 7422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1cde7204582a4d949ce0496273f863e4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1cde7204582a4d949ce0496273f863e4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1cde7204582a4d949ce0496273f863e4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42541 } } }
I20260808 22:31:03.367401 7422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1cde7204582a4d949ce0496273f863e4 [sys.catalog]: This master's current role is: LEADER
W20260808 22:31:03.371147 7445 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1cde7204582a4d949ce0496273f863e4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:31:03.371264 7445 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:31:03.371343 7431 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:31:03.372143 7431 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:31:03.377457 7431 catalog_manager.cc:1347] Generated new cluster ID: 28f3764a6c054438b55bc5648677da23
I20260808 22:31:03.377525 7431 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:31:03.401738 7431 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:31:03.403035 7431 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:31:03.411978 7431 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1cde7204582a4d949ce0496273f863e4: Generated new TSK 0
I20260808 22:31:03.412671 7431 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:31:03.422246 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 7266
W20260808 22:31:03.433444 7235 connection.cc:535] client connection to 127.0.0.1:42541 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42541): Connection reset by peer (error 104)
2026-08-08T22:31:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1665 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:31:03 dist-test-slave-7f01 krb5kdc[7451](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:31:03 dist-test-slave-7f01 krb5kdc[7451](info): set up 2 sockets
Aug 08 22:31:03 dist-test-slave-7f01 krb5kdc[7451](info): commencing operation
krb5kdc: starting...
W20260808 22:31:04.681329 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:31:04 dist-test-slave-7f01 krb5kdc[7451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228264, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:31:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:31:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 22:31:04.804739 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42779
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42779
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:04.917215 7467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:04.917469 7467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:04.917511 7467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:04.921407 7467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:31:04.921469 7467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:04.921489 7467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:04.921504 7467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:31:04.921518 7467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:31:04.926708 7467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42779
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42779
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:04.927647 7467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:04.928673 7467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:04.936802 7478 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:04.936962 7467 server_base.cc:1034] running on GCE node
W20260808 22:31:04.936982 7484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:04.937055 7481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:04.937783 7467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:04.938813 7467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:04.939982 7467 hybrid_clock.cc:648] HybridClock initialized: now 1786228264939962 us; error 30 us; skew 500 ppm
Aug 08 22:31:04 dist-test-slave-7f01 krb5kdc[7451](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228264, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:04.942647 7467 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 22:31:04.943709 7467 webserver.cc:460] Webserver started at http://127.0.0.1:37955/ using document root <none> and password file <none>
I20260808 22:31:04.944362 7467 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:04.944428 7467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:04.944639 7467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:04.946648 7467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "43bfb0b757f543dbafdd47da232eef39"
format_stamp: "Formatted at 2026-08-08 22:31:04 on dist-test-slave-7f01"
server_key: "92ecffd8321be978d40308f78d42d8d7"
server_key_iv: "e6a7bc97eac531212d51e8629e815980"
server_key_version: "encryptionkey@0"
I20260808 22:31:04.947255 7467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "43bfb0b757f543dbafdd47da232eef39"
format_stamp: "Formatted at 2026-08-08 22:31:04 on dist-test-slave-7f01"
server_key: "92ecffd8321be978d40308f78d42d8d7"
server_key_iv: "e6a7bc97eac531212d51e8629e815980"
server_key_version: "encryptionkey@0"
I20260808 22:31:04.950999 7467 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 22:31:04.953389 7493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:04.954536 7467 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:31:04.954679 7467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "43bfb0b757f543dbafdd47da232eef39"
format_stamp: "Formatted at 2026-08-08 22:31:04 on dist-test-slave-7f01"
server_key: "92ecffd8321be978d40308f78d42d8d7"
server_key_iv: "e6a7bc97eac531212d51e8629e815980"
server_key_version: "encryptionkey@0"
I20260808 22:31:04.954774 7467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:04.959235 7467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:04.962184 7467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:04.962378 7467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:04.969337 7467 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42779
I20260808 22:31:04.969364 7591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42779 every 8 connection(s)
I20260808 22:31:04.970239 7467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:31:04.971122 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 7467
I20260808 22:31:04.971287 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:31:04.971594 5167 external_mini_cluster.cc:1428] Setting key b8c6d5f21831c352fe2922dda768f2fd
I20260808 22:31:04.973377 7592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:04.977129 7603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:04.973028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.49:34275 (local address 127.0.0.1:42779)
0808 22:31:04.973457 (+ 429us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:04.973477 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:04.973518 (+ 41us) server_negotiation.cc:407] Connection header received
0808 22:31:04.974345 (+ 827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:04.974369 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:04.974709 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:04.975074 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:04.975756 (+ 682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:04.975763 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:04.975779 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:31:04.975791 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:04.975853 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:31:04.975883 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:04.976030 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:04.976098 (+ 68us) server_negotiation.cc:299] Negotiation successful
0808 22:31:04.976247 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":115,"threads_started":1}
I20260808 22:31:04.979888 5167 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:31:04.980268 7592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43bfb0b757f543dbafdd47da232eef39: Bootstrap starting.
I20260808 22:31:04.982615 7592 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 43bfb0b757f543dbafdd47da232eef39: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:04.983438 7603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:04.981925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.49:33221 (local address 127.0.0.1:42779)
0808 22:31:04.982038 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:04.982043 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:04.982239 (+ 196us) server_negotiation.cc:407] Connection header received
0808 22:31:04.982322 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:04.982325 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:04.982359 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:04.982436 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:04.983103 (+ 667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:04.983106 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:04.983109 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:31:04.983121 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:04.983147 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:31:04.983159 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:04.983251 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:04.983285 (+ 34us) server_negotiation.cc:299] Negotiation successful
0808 22:31:04.983322 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 22:31:04.983484 7592 log.cc:824] T 00000000000000000000000000000000 P 43bfb0b757f543dbafdd47da232eef39: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:04.985417 7592 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 43bfb0b757f543dbafdd47da232eef39: No bootstrap required, opened a new log
I20260808 22:31:04.988826 7592 raft_consensus.cc:348] T 00000000000000000000000000000000 P 43bfb0b757f543dbafdd47da232eef39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43bfb0b757f543dbafdd47da232eef39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42779 } }
I20260808 22:31:04.989034 7592 raft_consensus.cc:374] T 00000000000000000000000000000000 P 43bfb0b757f543dbafdd47da232eef39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:04.989078 7592 raft_consensus.cc:729] T 00000000000000000000000000000000 P 43bfb0b757f543dbafdd47da232eef39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 43bfb0b757f543dbafdd47da232eef39, State: Initialized, Role: FOLLOWER
I20260808 22:31:04.989420 7592 consensus_queue.cc:260] T 00000000000000000000000000000000 P 43bfb0b757f543dbafdd47da232eef39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43bfb0b757f543dbafdd47da232eef39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42779 } }
I20260808 22:31:04.989544 7592 raft_consensus.cc:388] T 00000000000000000000000000000000 P 43bfb0b757f543dbafdd47da232eef39 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:31:04.989629 7592 raft_consensus.cc:482] T 00000000000000000000000000000000 P 43bfb0b757f543dbafdd47da232eef39 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:31:04.989763 7592 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 43bfb0b757f543dbafdd47da232eef39 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:04.990582 7592 raft_consensus.cc:504] T 00000000000000000000000000000000 P 43bfb0b757f543dbafdd47da232eef39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43bfb0b757f543dbafdd47da232eef39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42779 } }
I20260808 22:31:04.990917 7592 leader_election.cc:302] T 00000000000000000000000000000000 P 43bfb0b757f543dbafdd47da232eef39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 43bfb0b757f543dbafdd47da232eef39; no voters:
I20260808 22:31:04.991209 7592 leader_election.cc:288] T 00000000000000000000000000000000 P 43bfb0b757f543dbafdd47da232eef39 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:31:04.991305 7622 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 43bfb0b757f543dbafdd47da232eef39 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:04.991554 7622 raft_consensus.cc:686] T 00000000000000000000000000000000 P 43bfb0b757f543dbafdd47da232eef39 [term 1 LEADER]: Becoming Leader. State: Replica: 43bfb0b757f543dbafdd47da232eef39, State: Running, Role: LEADER
I20260808 22:31:04.991874 7622 consensus_queue.cc:237] T 00000000000000000000000000000000 P 43bfb0b757f543dbafdd47da232eef39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43bfb0b757f543dbafdd47da232eef39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42779 } }
I20260808 22:31:04.992482 7592 sys_catalog.cc:564] T 00000000000000000000000000000000 P 43bfb0b757f543dbafdd47da232eef39 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:31:04.993646 7624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43bfb0b757f543dbafdd47da232eef39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 43bfb0b757f543dbafdd47da232eef39. Latest consensus state: current_term: 1 leader_uuid: "43bfb0b757f543dbafdd47da232eef39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43bfb0b757f543dbafdd47da232eef39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42779 } } }
I20260808 22:31:04.993780 7624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43bfb0b757f543dbafdd47da232eef39 [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:04.994133 7623 sys_catalog.cc:455] T 00000000000000000000000000000000 P 43bfb0b757f543dbafdd47da232eef39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "43bfb0b757f543dbafdd47da232eef39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "43bfb0b757f543dbafdd47da232eef39" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42779 } } }
I20260808 22:31:04.994226 7623 sys_catalog.cc:458] T 00000000000000000000000000000000 P 43bfb0b757f543dbafdd47da232eef39 [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:04.994313 7636 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:31:04.997313 7636 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:31:05.002259 7636 catalog_manager.cc:1347] Generated new cluster ID: a21d752b33cc4822ae4d68904ebf35ed
I20260808 22:31:05.002326 7636 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:31:05.015975 7636 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:31:05.016286 7636 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:31:05.026397 7636 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 43bfb0b757f543dbafdd47da232eef39: Generated new TSK 0
I20260808 22:31:05.027154 7636 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:31:05.030160 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 7467
2026-08-08T22:31:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1610 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:31:05 dist-test-slave-7f01 krb5kdc[7654](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:31:05 dist-test-slave-7f01 krb5kdc[7654](info): set up 2 sockets
Aug 08 22:31:05 dist-test-slave-7f01 krb5kdc[7654](info): commencing operation
krb5kdc: starting...
W20260808 22:31:06.301787 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.243s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:31:06 dist-test-slave-7f01 krb5kdc[7654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:31:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:31:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 22:31:06.426419 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35411
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35411
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:06.537966 7679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:06.538216 7679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:06.538257 7679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:06.542862 7679 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:31:06.542918 7679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:06.542937 7679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:06.542954 7679 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:31:06.543015 7679 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:31:06.548198 7679 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35411
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:06.549226 7679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:06.550243 7679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:06.558732 7686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:06.558740 7689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:06.558725 7685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:06.559055 7679 server_base.cc:1034] running on GCE node
I20260808 22:31:06.559657 7679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:06.560741 7679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:06.561931 7679 hybrid_clock.cc:648] HybridClock initialized: now 1786228266561913 us; error 29 us; skew 500 ppm
Aug 08 22:31:06 dist-test-slave-7f01 krb5kdc[7654](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228266, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:06.564863 7679 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 22:31:06.565739 7679 webserver.cc:460] Webserver started at http://127.0.0.1:40023/ using document root <none> and password file <none>
I20260808 22:31:06.566298 7679 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:06.566361 7679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:06.566625 7679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:06.568724 7679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "97d766be02d0466a8821f3e2d4339477"
format_stamp: "Formatted at 2026-08-08 22:31:06 on dist-test-slave-7f01"
server_key: "a2ffe05ecb438ab0411b9e1c1585f96d"
server_key_iv: "cd98bbaa6467b173bb1a5d822c2ad2f2"
server_key_version: "encryptionkey@0"
I20260808 22:31:06.569278 7679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "97d766be02d0466a8821f3e2d4339477"
format_stamp: "Formatted at 2026-08-08 22:31:06 on dist-test-slave-7f01"
server_key: "a2ffe05ecb438ab0411b9e1c1585f96d"
server_key_iv: "cd98bbaa6467b173bb1a5d822c2ad2f2"
server_key_version: "encryptionkey@0"
I20260808 22:31:06.572939 7679 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:31:06.575295 7699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:06.576529 7679 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:31:06.576681 7679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "97d766be02d0466a8821f3e2d4339477"
format_stamp: "Formatted at 2026-08-08 22:31:06 on dist-test-slave-7f01"
server_key: "a2ffe05ecb438ab0411b9e1c1585f96d"
server_key_iv: "cd98bbaa6467b173bb1a5d822c2ad2f2"
server_key_version: "encryptionkey@0"
I20260808 22:31:06.576794 7679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:06.591262 7679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:06.594136 7679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:06.594347 7679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:06.601354 7679 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35411
I20260808 22:31:06.601420 7797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35411 every 8 connection(s)
I20260808 22:31:06.602331 7679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:31:06.602543 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 7679
I20260808 22:31:06.602677 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:31:06.603045 5167 external_mini_cluster.cc:1428] Setting key 88d5ca74e169a09a6b31b4363fafd347
I20260808 22:31:06.605857 7800 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:06.612461 7800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97d766be02d0466a8821f3e2d4339477: Bootstrap starting.
Aug 08 22:31:06 dist-test-slave-7f01 krb5kdc[7654](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228266, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260808 22:31:06.615260 7800 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97d766be02d0466a8821f3e2d4339477: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:06.616259 7800 log.cc:824] T 00000000000000000000000000000000 P 97d766be02d0466a8821f3e2d4339477: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:06.618219 7800 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97d766be02d0466a8821f3e2d4339477: No bootstrap required, opened a new log
I20260808 22:31:06.618219 7804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:06.604411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.49:35259 (local address 127.0.0.1:35411)
0808 22:31:06.604824 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:06.604843 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:06.604885 (+ 42us) server_negotiation.cc:407] Connection header received
0808 22:31:06.605767 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:06.605790 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:06.606212 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:06.606565 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:06.607258 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:06.608815 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:06.609633 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:06.610318 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:06.610477 (+ 159us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:06.613164 (+ 2687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:06.613171 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:06.613186 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:06.613216 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:06.615086 (+ 1870us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:06.615635 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:06.615641 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:06.615647 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:06.615808 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:06.616141 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:06.616144 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:06.616146 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:06.616716 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:06.616965 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:06.617299 (+ 334us) server_negotiation.cc:299] Negotiation successful
0808 22:31:06.617527 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":122,"threads_started":1}
I20260808 22:31:06.621313 5167 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:31:06.622323 7800 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97d766be02d0466a8821f3e2d4339477 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97d766be02d0466a8821f3e2d4339477" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35411 } }
I20260808 22:31:06.622632 7800 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97d766be02d0466a8821f3e2d4339477 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:06.622717 7800 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97d766be02d0466a8821f3e2d4339477 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97d766be02d0466a8821f3e2d4339477, State: Initialized, Role: FOLLOWER
I20260808 22:31:06.623193 7800 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97d766be02d0466a8821f3e2d4339477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97d766be02d0466a8821f3e2d4339477" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35411 } }
I20260808 22:31:06.623411 7800 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97d766be02d0466a8821f3e2d4339477 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:31:06.623524 7800 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97d766be02d0466a8821f3e2d4339477 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:31:06.623658 7800 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97d766be02d0466a8821f3e2d4339477 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:06.624564 7800 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97d766be02d0466a8821f3e2d4339477 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97d766be02d0466a8821f3e2d4339477" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35411 } }
I20260808 22:31:06.624903 7800 leader_election.cc:302] T 00000000000000000000000000000000 P 97d766be02d0466a8821f3e2d4339477 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97d766be02d0466a8821f3e2d4339477; no voters:
I20260808 22:31:06.625247 7800 leader_election.cc:288] T 00000000000000000000000000000000 P 97d766be02d0466a8821f3e2d4339477 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:31:06.625358 7820 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97d766be02d0466a8821f3e2d4339477 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:06.625576 7820 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97d766be02d0466a8821f3e2d4339477 [term 1 LEADER]: Becoming Leader. State: Replica: 97d766be02d0466a8821f3e2d4339477, State: Running, Role: LEADER
I20260808 22:31:06.625968 7820 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97d766be02d0466a8821f3e2d4339477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97d766be02d0466a8821f3e2d4339477" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35411 } }
I20260808 22:31:06.626466 7800 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97d766be02d0466a8821f3e2d4339477 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:31:06.627244 7820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97d766be02d0466a8821f3e2d4339477 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97d766be02d0466a8821f3e2d4339477. Latest consensus state: current_term: 1 leader_uuid: "97d766be02d0466a8821f3e2d4339477" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97d766be02d0466a8821f3e2d4339477" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35411 } } }
I20260808 22:31:06.627385 7820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97d766be02d0466a8821f3e2d4339477 [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:06.627907 7834 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:31:06.628968 7821 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97d766be02d0466a8821f3e2d4339477 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97d766be02d0466a8821f3e2d4339477" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97d766be02d0466a8821f3e2d4339477" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35411 } } }
I20260808 22:31:06.629084 7821 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97d766be02d0466a8821f3e2d4339477 [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:06.631217 7834 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:31:06.632929 7804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:06.623401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.49:55735 (local address 127.0.0.1:35411)
0808 22:31:06.623505 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:06.623509 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:06.623779 (+ 270us) server_negotiation.cc:407] Connection header received
0808 22:31:06.623949 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:06.623952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:06.623994 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:06.624107 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:06.624815 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:06.625504 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:06.626658 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:06.627015 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:06.627119 (+ 104us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:06.629119 (+ 2000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:06.629126 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:06.629129 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:06.629156 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:06.630877 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:06.631516 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:06.631519 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:06.631521 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:06.631607 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:06.632057 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:06.632060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:06.632061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:06.632215 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:06.632378 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:06.632673 (+ 295us) server_negotiation.cc:299] Negotiation successful
0808 22:31:06.632762 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260808 22:31:06.637084 7834 catalog_manager.cc:1347] Generated new cluster ID: 76146bf4fa2945df9ad5d44941f047a0
I20260808 22:31:06.637151 7834 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:31:06.646593 7834 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:31:06.647722 7834 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:31:06.658717 7834 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97d766be02d0466a8821f3e2d4339477: Generated new TSK 0
I20260808 22:31:06.659472 7834 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:31:06.669440 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 7679
W20260808 22:31:06.684011 7652 connection.cc:535] client connection to 127.0.0.1:35411 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35411): Connection reset by peer (error 104)
2026-08-08T22:31:06Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1640 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:31:06 dist-test-slave-7f01 krb5kdc[7854](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:31:06 dist-test-slave-7f01 krb5kdc[7854](info): set up 2 sockets
Aug 08 22:31:06 dist-test-slave-7f01 krb5kdc[7854](info): commencing operation
krb5kdc: starting...
W20260808 22:31:07.933610 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.235s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:31:08 dist-test-slave-7f01 krb5kdc[7854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228268, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:31:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:31:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 22:31:08.058481 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45853
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46857
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45853
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:08.174372 7877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:08.174603 7877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:08.174642 7877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:08.178571 7877 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:31:08.178625 7877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:08.178645 7877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:08.178661 7877 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:31:08.178715 7877 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:31:08.183770 7877 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46857
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45853
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:08.184705 7877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:08.185674 7877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:08.194124 7891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:08.194141 7886 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:08.194162 7889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:08.194398 7877 server_base.cc:1034] running on GCE node
I20260808 22:31:08.195001 7877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:08.196105 7877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:08.197299 7877 hybrid_clock.cc:648] HybridClock initialized: now 1786228268197268 us; error 42 us; skew 500 ppm
Aug 08 22:31:08 dist-test-slave-7f01 krb5kdc[7854](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228268, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:08.200058 7877 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 22:31:08.200929 7877 webserver.cc:460] Webserver started at http://127.0.0.1:37985/ using document root <none> and password file <none>
I20260808 22:31:08.201498 7877 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:08.201593 7877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:08.201833 7877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:08.203812 7877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "3423755f65884fec9a4851c66c46db46"
format_stamp: "Formatted at 2026-08-08 22:31:08 on dist-test-slave-7f01"
server_key: "9509cf018e719031dc7960a2d040ffa8"
server_key_iv: "122a5da52981bd49b5d6c3565c53f354"
server_key_version: "encryptionkey@0"
I20260808 22:31:08.204396 7877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "3423755f65884fec9a4851c66c46db46"
format_stamp: "Formatted at 2026-08-08 22:31:08 on dist-test-slave-7f01"
server_key: "9509cf018e719031dc7960a2d040ffa8"
server_key_iv: "122a5da52981bd49b5d6c3565c53f354"
server_key_version: "encryptionkey@0"
I20260808 22:31:08.208194 7877 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 22:31:08.210594 7908 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:08.211896 7877 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:31:08.212037 7877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "3423755f65884fec9a4851c66c46db46"
format_stamp: "Formatted at 2026-08-08 22:31:08 on dist-test-slave-7f01"
server_key: "9509cf018e719031dc7960a2d040ffa8"
server_key_iv: "122a5da52981bd49b5d6c3565c53f354"
server_key_version: "encryptionkey@0"
I20260808 22:31:08.212150 7877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:08.225556 7877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:08.228523 7877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:08.228717 7877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:08.235859 7877 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45853
I20260808 22:31:08.235867 7995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45853 every 8 connection(s)
I20260808 22:31:08.236774 7877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:31:08.240023 7996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:08.245158 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 7877
I20260808 22:31:08.245325 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:31:08.245635 5167 external_mini_cluster.cc:1428] Setting key bf23e52ba45bba1bf6534a88fa6ad582
I20260808 22:31:08.245955 7996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3423755f65884fec9a4851c66c46db46: Bootstrap starting.
I20260808 22:31:08.248617 7996 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3423755f65884fec9a4851c66c46db46: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:08.249351 7996 log.cc:824] T 00000000000000000000000000000000 P 3423755f65884fec9a4851c66c46db46: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:08.251210 7996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3423755f65884fec9a4851c66c46db46: No bootstrap required, opened a new log
I20260808 22:31:08.254760 8007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:08.247072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.49:51123 (local address 127.0.0.1:45853)
0808 22:31:08.247528 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:08.247543 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:08.247577 (+ 34us) server_negotiation.cc:407] Connection header received
0808 22:31:08.248396 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:08.248418 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:08.248825 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:08.249163 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:08.249616 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:08.250562 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:08.251538 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:08.252078 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:08.252254 (+ 176us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:08.253165 (+ 911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:08.253170 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:08.253181 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:31:08.253547 (+ 366us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:08.253599 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:31:08.253621 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:08.253762 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:08.253946 (+ 184us) server_negotiation.cc:299] Negotiation successful
0808 22:31:08.254084 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":148,"threads_started":1}
I20260808 22:31:08.255079 7996 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3423755f65884fec9a4851c66c46db46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3423755f65884fec9a4851c66c46db46" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45853 } }
I20260808 22:31:08.255347 7996 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3423755f65884fec9a4851c66c46db46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:08.255417 7996 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3423755f65884fec9a4851c66c46db46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3423755f65884fec9a4851c66c46db46, State: Initialized, Role: FOLLOWER
I20260808 22:31:08.255895 7996 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3423755f65884fec9a4851c66c46db46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3423755f65884fec9a4851c66c46db46" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45853 } }
I20260808 22:31:08.256065 7996 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3423755f65884fec9a4851c66c46db46 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:31:08.256155 7996 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3423755f65884fec9a4851c66c46db46 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:31:08.256280 7996 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3423755f65884fec9a4851c66c46db46 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:08.257370 7996 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3423755f65884fec9a4851c66c46db46 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3423755f65884fec9a4851c66c46db46" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45853 } }
I20260808 22:31:08.257745 5167 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:31:08.257746 7996 leader_election.cc:302] T 00000000000000000000000000000000 P 3423755f65884fec9a4851c66c46db46 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3423755f65884fec9a4851c66c46db46; no voters:
I20260808 22:31:08.258102 7996 leader_election.cc:288] T 00000000000000000000000000000000 P 3423755f65884fec9a4851c66c46db46 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:31:08.258160 8010 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3423755f65884fec9a4851c66c46db46 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:08.258352 8010 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3423755f65884fec9a4851c66c46db46 [term 1 LEADER]: Becoming Leader. State: Replica: 3423755f65884fec9a4851c66c46db46, State: Running, Role: LEADER
I20260808 22:31:08.258704 8010 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3423755f65884fec9a4851c66c46db46 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3423755f65884fec9a4851c66c46db46" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45853 } }
I20260808 22:31:08.259330 7996 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3423755f65884fec9a4851c66c46db46 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:31:08.260442 8014 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3423755f65884fec9a4851c66c46db46 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3423755f65884fec9a4851c66c46db46. Latest consensus state: current_term: 1 leader_uuid: "3423755f65884fec9a4851c66c46db46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3423755f65884fec9a4851c66c46db46" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45853 } } }
I20260808 22:31:08.260571 8014 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3423755f65884fec9a4851c66c46db46 [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:08.260907 8011 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3423755f65884fec9a4851c66c46db46 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3423755f65884fec9a4851c66c46db46" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3423755f65884fec9a4851c66c46db46" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45853 } } }
I20260808 22:31:08.260977 8035 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:31:08.261034 8011 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3423755f65884fec9a4851c66c46db46 [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:08.264159 8035 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:31:08.266031 8007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:08.260246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.49:53501 (local address 127.0.0.1:45853)
0808 22:31:08.260347 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:08.260350 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:08.261889 (+ 1539us) server_negotiation.cc:407] Connection header received
0808 22:31:08.261982 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:08.261985 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:08.262029 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:08.262109 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:08.262384 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:08.262907 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:08.264121 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:08.264406 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:08.264520 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:08.265462 (+ 942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:08.265469 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:08.265473 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:31:08.265489 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:08.265526 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:31:08.265538 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:08.265601 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:08.265831 (+ 230us) server_negotiation.cc:299] Negotiation successful
0808 22:31:08.265868 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 22:31:08.270249 8035 catalog_manager.cc:1347] Generated new cluster ID: 280c037dd598458399770ed2c5d0b00f
I20260808 22:31:08.270313 8035 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:31:08.281737 8035 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:31:08.282621 8035 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:31:08.293433 8035 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3423755f65884fec9a4851c66c46db46: Generated new TSK 0
I20260808 22:31:08.294080 8035 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:31:08.301309 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 7877
W20260808 22:31:08.319486 7850 connection.cc:535] client connection to 127.0.0.1:45853 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45853): Connection reset by peer (error 104)
2026-08-08T22:31:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1635 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:31:08 dist-test-slave-7f01 krb5kdc[8058](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:31:08 dist-test-slave-7f01 krb5kdc[8058](info): set up 2 sockets
Aug 08 22:31:08 dist-test-slave-7f01 krb5kdc[8058](info): commencing operation
krb5kdc: starting...
W20260808 22:31:09.560325 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.226s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:31:09 dist-test-slave-7f01 krb5kdc[8058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:31:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:31:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260808 22:31:09.685233 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37221
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37221
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:09.795630 8086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:09.795888 8086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:09.795928 8086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:09.799722 8086 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:31:09.799783 8086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:09.799803 8086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:09.799818 8086 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:31:09.799834 8086 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:31:09.804744 8086 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37221
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786227991988852-5167-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37221
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:09.805632 8086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:09.806576 8086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:09.814860 8101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:09.814957 8086 server_base.cc:1034] running on GCE node
W20260808 22:31:09.814860 8094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:09.814869 8092 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:09.815722 8086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:09.816856 8086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:09.818024 8086 hybrid_clock.cc:648] HybridClock initialized: now 1786228269818000 us; error 47 us; skew 500 ppm
Aug 08 22:31:09 dist-test-slave-7f01 krb5kdc[8058](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228269, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:09.820851 8086 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260808 22:31:09.821751 8086 webserver.cc:460] Webserver started at http://127.0.0.1:40455/ using document root <none> and password file <none>
I20260808 22:31:09.822322 8086 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:09.822417 8086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:09.822654 8086 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:09.824728 8086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "243c791bd7bf41e4bd1c5c823e7c94a4"
format_stamp: "Formatted at 2026-08-08 22:31:09 on dist-test-slave-7f01"
server_key: "02af0299d0da23d06215271f32ef73ba"
server_key_iv: "cee33ef2a4b9cd486adeeca8bccebf72"
server_key_version: "encryptionkey@0"
I20260808 22:31:09.825285 8086 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "243c791bd7bf41e4bd1c5c823e7c94a4"
format_stamp: "Formatted at 2026-08-08 22:31:09 on dist-test-slave-7f01"
server_key: "02af0299d0da23d06215271f32ef73ba"
server_key_iv: "cee33ef2a4b9cd486adeeca8bccebf72"
server_key_version: "encryptionkey@0"
I20260808 22:31:09.828935 8086 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:31:09.831256 8114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:09.832499 8086 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 22:31:09.832654 8086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "243c791bd7bf41e4bd1c5c823e7c94a4"
format_stamp: "Formatted at 2026-08-08 22:31:09 on dist-test-slave-7f01"
server_key: "02af0299d0da23d06215271f32ef73ba"
server_key_iv: "cee33ef2a4b9cd486adeeca8bccebf72"
server_key_version: "encryptionkey@0"
I20260808 22:31:09.832763 8086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:09.837324 8086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:09.840204 8086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:09.840394 8086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:09.847085 8086 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37221
I20260808 22:31:09.847098 8208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37221 every 8 connection(s)
I20260808 22:31:09.848042 8086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:31:09.850684 8209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:09.851015 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 8086
I20260808 22:31:09.851171 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:31:09.851455 5167 external_mini_cluster.cc:1428] Setting key 288528b3faf009fa483f0d3518c55990
I20260808 22:31:09.856393 8214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:09.852798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.49:35045 (local address 127.0.0.1:37221)
0808 22:31:09.853193 (+ 395us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:09.853207 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:09.853240 (+ 33us) server_negotiation.cc:407] Connection header received
0808 22:31:09.853880 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:09.853896 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:09.854162 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:09.854424 (+ 262us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:09.855237 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:09.855242 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:09.855253 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:31:09.855265 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:09.855315 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:31:09.855333 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:09.855438 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:09.855548 (+ 110us) server_negotiation.cc:299] Negotiation successful
0808 22:31:09.855711 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":125,"threads_started":1}
I20260808 22:31:09.857414 8209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 243c791bd7bf41e4bd1c5c823e7c94a4: Bootstrap starting.
I20260808 22:31:09.859194 5167 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:31:09.859997 8209 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 243c791bd7bf41e4bd1c5c823e7c94a4: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:09.860767 8209 log.cc:824] T 00000000000000000000000000000000 P 243c791bd7bf41e4bd1c5c823e7c94a4: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:09.862675 8209 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 243c791bd7bf41e4bd1c5c823e7c94a4: No bootstrap required, opened a new log
I20260808 22:31:09.863008 8214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:09.861320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.49:37441 (local address 127.0.0.1:37221)
0808 22:31:09.861455 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:09.861460 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:09.861620 (+ 160us) server_negotiation.cc:407] Connection header received
0808 22:31:09.861750 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:09.861753 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:09.861792 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:09.861882 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:09.862576 (+ 694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:09.862579 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:09.862582 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:31:09.862593 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:09.862620 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:31:09.862639 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:09.862728 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:09.862842 (+ 114us) server_negotiation.cc:299] Negotiation successful
0808 22:31:09.862888 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260808 22:31:09.866417 8209 raft_consensus.cc:348] T 00000000000000000000000000000000 P 243c791bd7bf41e4bd1c5c823e7c94a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "243c791bd7bf41e4bd1c5c823e7c94a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37221 } }
I20260808 22:31:09.866641 8209 raft_consensus.cc:374] T 00000000000000000000000000000000 P 243c791bd7bf41e4bd1c5c823e7c94a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:09.866715 8209 raft_consensus.cc:729] T 00000000000000000000000000000000 P 243c791bd7bf41e4bd1c5c823e7c94a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 243c791bd7bf41e4bd1c5c823e7c94a4, State: Initialized, Role: FOLLOWER
I20260808 22:31:09.867190 8209 consensus_queue.cc:260] T 00000000000000000000000000000000 P 243c791bd7bf41e4bd1c5c823e7c94a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "243c791bd7bf41e4bd1c5c823e7c94a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37221 } }
I20260808 22:31:09.867332 8209 raft_consensus.cc:388] T 00000000000000000000000000000000 P 243c791bd7bf41e4bd1c5c823e7c94a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:31:09.867414 8209 raft_consensus.cc:482] T 00000000000000000000000000000000 P 243c791bd7bf41e4bd1c5c823e7c94a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:31:09.867537 8209 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 243c791bd7bf41e4bd1c5c823e7c94a4 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:09.868571 8209 raft_consensus.cc:504] T 00000000000000000000000000000000 P 243c791bd7bf41e4bd1c5c823e7c94a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "243c791bd7bf41e4bd1c5c823e7c94a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37221 } }
I20260808 22:31:09.868911 8209 leader_election.cc:302] T 00000000000000000000000000000000 P 243c791bd7bf41e4bd1c5c823e7c94a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 243c791bd7bf41e4bd1c5c823e7c94a4; no voters:
I20260808 22:31:09.869251 8209 leader_election.cc:288] T 00000000000000000000000000000000 P 243c791bd7bf41e4bd1c5c823e7c94a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:31:09.869407 8227 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 243c791bd7bf41e4bd1c5c823e7c94a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:09.869668 8227 raft_consensus.cc:686] T 00000000000000000000000000000000 P 243c791bd7bf41e4bd1c5c823e7c94a4 [term 1 LEADER]: Becoming Leader. State: Replica: 243c791bd7bf41e4bd1c5c823e7c94a4, State: Running, Role: LEADER
I20260808 22:31:09.869994 8227 consensus_queue.cc:237] T 00000000000000000000000000000000 P 243c791bd7bf41e4bd1c5c823e7c94a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "243c791bd7bf41e4bd1c5c823e7c94a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37221 } }
I20260808 22:31:09.870680 8209 sys_catalog.cc:564] T 00000000000000000000000000000000 P 243c791bd7bf41e4bd1c5c823e7c94a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:31:09.871805 8230 sys_catalog.cc:455] T 00000000000000000000000000000000 P 243c791bd7bf41e4bd1c5c823e7c94a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 243c791bd7bf41e4bd1c5c823e7c94a4. Latest consensus state: current_term: 1 leader_uuid: "243c791bd7bf41e4bd1c5c823e7c94a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "243c791bd7bf41e4bd1c5c823e7c94a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37221 } } }
I20260808 22:31:09.871953 8230 sys_catalog.cc:458] T 00000000000000000000000000000000 P 243c791bd7bf41e4bd1c5c823e7c94a4 [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:09.872298 8229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 243c791bd7bf41e4bd1c5c823e7c94a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "243c791bd7bf41e4bd1c5c823e7c94a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "243c791bd7bf41e4bd1c5c823e7c94a4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37221 } } }
I20260808 22:31:09.872392 8245 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:31:09.872478 8229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 243c791bd7bf41e4bd1c5c823e7c94a4 [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:09.875615 8245 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:31:09.881412 8245 catalog_manager.cc:1347] Generated new cluster ID: 788239b452a1414e8320f81b05701cb7
I20260808 22:31:09.881480 8245 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:31:09.893625 8245 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:31:09.893940 8245 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:31:09.902485 8245 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 243c791bd7bf41e4bd1c5c823e7c94a4: Generated new TSK 0
I20260808 22:31:09.903160 8245 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:31:09.911753 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 8086
2026-08-08T22:31:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1604 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:31:09 dist-test-slave-7f01 krb5kdc[8274](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:31:09 dist-test-slave-7f01 krb5kdc[8274](info): set up 2 sockets
Aug 08 22:31:09 dist-test-slave-7f01 krb5kdc[8274](info): commencing operation
krb5kdc: starting...
W20260808 22:31:11.180401 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:31:11 dist-test-slave-7f01 krb5kdc[8274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228271, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:31:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:31:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.11.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
I20260808 22:31:11.305105 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:44415
--webserver_interface=127.5.11.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:43853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:44415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:11.413929 8303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:11.414218 8303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:11.414290 8303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:11.418200 8303 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:31:11.418289 8303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:11.418330 8303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:11.418381 8303 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:31:11.418421 8303 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:31:11.423519 8303 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:43853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:44415
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:44415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:11.424556 8303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:11.425535 8303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:11.434002 8309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:11.434021 8312 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:11.434021 8319 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:11.434176 8303 server_base.cc:1034] running on GCE node
I20260808 22:31:11.434862 8303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:11.435901 8303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:11.437063 8303 hybrid_clock.cc:648] HybridClock initialized: now 1786228271437046 us; error 30 us; skew 500 ppm
Aug 08 22:31:11 dist-test-slave-7f01 krb5kdc[8274](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228271, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:11.439796 8303 init.cc:377] Logged in from keytab as kudu/127.5.11.254@KRBTEST.COM (short username kudu)
I20260808 22:31:11.440738 8303 webserver.cc:460] Webserver started at http://127.5.11.254:43577/ using document root <none> and password file <none>
I20260808 22:31:11.441275 8303 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:11.441335 8303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:11.441584 8303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:11.443542 8303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "871d459b8fe9405eb5599c0790b6f91a"
format_stamp: "Formatted at 2026-08-08 22:31:11 on dist-test-slave-7f01"
server_key: "d86d7ad7d7ae2a129bf8a302d68d1c10"
server_key_iv: "484d7017bd557929c0448620ae563f3f"
server_key_version: "encryptionkey@0"
I20260808 22:31:11.444140 8303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "871d459b8fe9405eb5599c0790b6f91a"
format_stamp: "Formatted at 2026-08-08 22:31:11 on dist-test-slave-7f01"
server_key: "d86d7ad7d7ae2a129bf8a302d68d1c10"
server_key_iv: "484d7017bd557929c0448620ae563f3f"
server_key_version: "encryptionkey@0"
I20260808 22:31:11.447678 8303 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:31:11.449994 8333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:11.451164 8303 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260808 22:31:11.451296 8303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "871d459b8fe9405eb5599c0790b6f91a"
format_stamp: "Formatted at 2026-08-08 22:31:11 on dist-test-slave-7f01"
server_key: "d86d7ad7d7ae2a129bf8a302d68d1c10"
server_key_iv: "484d7017bd557929c0448620ae563f3f"
server_key_version: "encryptionkey@0"
I20260808 22:31:11.451402 8303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:11.487002 8303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:11.489976 8303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:11.490190 8303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:11.497458 8303 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:44415
I20260808 22:31:11.497488 8437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:44415 every 8 connection(s)
I20260808 22:31:11.498428 8303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:31:11.501209 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 8303
I20260808 22:31:11.501379 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:31:11.501417 8439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:11.501735 5167 external_mini_cluster.cc:1428] Setting key f24750fdfd840038b1d28928fca7363a
I20260808 22:31:11.507880 8439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 871d459b8fe9405eb5599c0790b6f91a: Bootstrap starting.
I20260808 22:31:11.510228 8439 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 871d459b8fe9405eb5599c0790b6f91a: Neither blocks nor log segments found. Creating new log.
Aug 08 22:31:11 dist-test-slave-7f01 krb5kdc[8274](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228271, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:31:11.511010 8439 log.cc:824] T 00000000000000000000000000000000 P 871d459b8fe9405eb5599c0790b6f91a: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:11.513182 8439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 871d459b8fe9405eb5599c0790b6f91a: No bootstrap required, opened a new log
I20260808 22:31:11.515734 8445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:11.503149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38091 (local address 127.5.11.254:44415)
0808 22:31:11.503601 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:11.503615 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:11.503650 (+ 35us) server_negotiation.cc:407] Connection header received
0808 22:31:11.504307 (+ 657us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:11.504324 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:11.504662 (+ 338us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:11.504993 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:11.505814 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:11.506803 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:11.507676 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:11.508251 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:11.508419 (+ 168us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:11.510940 (+ 2521us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:11.510945 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:11.510959 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:11.510986 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:11.512854 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:11.513389 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:11.513396 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:11.513400 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:11.513513 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:11.513859 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:11.513869 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:11.513871 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:11.514248 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:11.514479 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:11.514803 (+ 324us) server_negotiation.cc:299] Negotiation successful
0808 22:31:11.515037 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":150,"threads_started":1}
I20260808 22:31:11.516971 8439 raft_consensus.cc:348] T 00000000000000000000000000000000 P 871d459b8fe9405eb5599c0790b6f91a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "871d459b8fe9405eb5599c0790b6f91a" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 44415 } }
I20260808 22:31:11.517252 8439 raft_consensus.cc:374] T 00000000000000000000000000000000 P 871d459b8fe9405eb5599c0790b6f91a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:11.517309 8439 raft_consensus.cc:729] T 00000000000000000000000000000000 P 871d459b8fe9405eb5599c0790b6f91a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 871d459b8fe9405eb5599c0790b6f91a, State: Initialized, Role: FOLLOWER
I20260808 22:31:11.517804 8439 consensus_queue.cc:260] T 00000000000000000000000000000000 P 871d459b8fe9405eb5599c0790b6f91a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "871d459b8fe9405eb5599c0790b6f91a" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 44415 } }
I20260808 22:31:11.517963 8439 raft_consensus.cc:388] T 00000000000000000000000000000000 P 871d459b8fe9405eb5599c0790b6f91a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:31:11.518060 8439 raft_consensus.cc:482] T 00000000000000000000000000000000 P 871d459b8fe9405eb5599c0790b6f91a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:31:11.518162 8439 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 871d459b8fe9405eb5599c0790b6f91a [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:11.518962 5167 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:31:11.519316 8439 raft_consensus.cc:504] T 00000000000000000000000000000000 P 871d459b8fe9405eb5599c0790b6f91a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "871d459b8fe9405eb5599c0790b6f91a" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 44415 } }
I20260808 22:31:11.519663 8439 leader_election.cc:302] T 00000000000000000000000000000000 P 871d459b8fe9405eb5599c0790b6f91a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 871d459b8fe9405eb5599c0790b6f91a; no voters:
I20260808 22:31:11.520053 8439 leader_election.cc:288] T 00000000000000000000000000000000 P 871d459b8fe9405eb5599c0790b6f91a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:31:11.520133 8448 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 871d459b8fe9405eb5599c0790b6f91a [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:11.520344 8448 raft_consensus.cc:686] T 00000000000000000000000000000000 P 871d459b8fe9405eb5599c0790b6f91a [term 1 LEADER]: Becoming Leader. State: Replica: 871d459b8fe9405eb5599c0790b6f91a, State: Running, Role: LEADER
I20260808 22:31:11.520705 8448 consensus_queue.cc:237] T 00000000000000000000000000000000 P 871d459b8fe9405eb5599c0790b6f91a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "871d459b8fe9405eb5599c0790b6f91a" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 44415 } }
I20260808 22:31:11.521359 8439 sys_catalog.cc:564] T 00000000000000000000000000000000 P 871d459b8fe9405eb5599c0790b6f91a [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:31:11.522394 8455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 871d459b8fe9405eb5599c0790b6f91a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 871d459b8fe9405eb5599c0790b6f91a. Latest consensus state: current_term: 1 leader_uuid: "871d459b8fe9405eb5599c0790b6f91a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "871d459b8fe9405eb5599c0790b6f91a" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 44415 } } }
I20260808 22:31:11.522563 8455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 871d459b8fe9405eb5599c0790b6f91a [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:11.522971 8456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 871d459b8fe9405eb5599c0790b6f91a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "871d459b8fe9405eb5599c0790b6f91a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "871d459b8fe9405eb5599c0790b6f91a" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 44415 } } }
I20260808 22:31:11.523128 8456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 871d459b8fe9405eb5599c0790b6f91a [sys.catalog]: This master's current role is: LEADER
W20260808 22:31:11.526546 8479 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 871d459b8fe9405eb5599c0790b6f91a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:31:11.526659 8479 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:31:11.526819 8470 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:31:11.527712 8470 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:31:11.530659 8445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:11.521066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37981 (local address 127.5.11.254:44415)
0808 22:31:11.521178 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:11.521182 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:11.521655 (+ 473us) server_negotiation.cc:407] Connection header received
0808 22:31:11.521786 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:11.521789 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:11.521835 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:11.521924 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:11.522937 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:11.523713 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:11.524668 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:11.524951 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:11.525082 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:11.526693 (+ 1611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:11.526697 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:11.526700 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:11.526725 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:11.528440 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:11.529018 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:11.529022 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:11.529024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:11.529117 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:11.529551 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:11.529554 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:11.529556 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:11.529759 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:11.529907 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:11.530246 (+ 339us) server_negotiation.cc:299] Negotiation successful
0808 22:31:11.530510 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 22:31:11.533893 8470 catalog_manager.cc:1347] Generated new cluster ID: d6e39851f0304eeeb7c782777f4fd61d
I20260808 22:31:11.533967 8470 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:31:11.551388 8470 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:31:11.552541 8470 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:31:11.558580 8470 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 871d459b8fe9405eb5599c0790b6f91a: Generated new TSK 0
I20260808 22:31:11.559274 8470 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:31:11.563642 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 8303
W20260808 22:31:11.573596 8271 connection.cc:535] client connection to 127.5.11.254:44415 recv error: Network error: failed to read from TLS socket (remote: 127.5.11.254:44415): Connection reset by peer (error 104)
2026-08-08T22:31:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1649 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:31:11 dist-test-slave-7f01 krb5kdc[8489](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:31:11 dist-test-slave-7f01 krb5kdc[8489](info): set up 2 sockets
Aug 08 22:31:11 dist-test-slave-7f01 krb5kdc[8489](info): commencing operation
krb5kdc: starting...
W20260808 22:31:12.828933 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.241s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:31:12 dist-test-slave-7f01 krb5kdc[8489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:31:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:31:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.11.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
I20260808 22:31:12.954851 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:45135
--webserver_interface=127.5.11.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:37615
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:45135
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:13.066727 8509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:13.067029 8509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:13.067070 8509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:13.071040 8509 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:31:13.071099 8509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:13.071118 8509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:13.071134 8509 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:31:13.071149 8509 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:31:13.076251 8509 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:37615
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:45135
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:45135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:13.077243 8509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:13.078236 8509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:13.086658 8524 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:13.086742 8509 server_base.cc:1034] running on GCE node
W20260808 22:31:13.086676 8519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:13.086681 8521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:13.087503 8509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:13.088598 8509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:13.089761 8509 hybrid_clock.cc:648] HybridClock initialized: now 1786228273089742 us; error 31 us; skew 500 ppm
Aug 08 22:31:13 dist-test-slave-7f01 krb5kdc[8489](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228273, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:13.092553 8509 init.cc:377] Logged in from keytab as kudu/127.5.11.254@KRBTEST.COM (short username kudu)
I20260808 22:31:13.093500 8509 webserver.cc:460] Webserver started at http://127.5.11.254:33093/ using document root <none> and password file <none>
I20260808 22:31:13.094077 8509 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:13.094172 8509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:13.094413 8509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:13.096601 8509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "439d32e08064471da6d702da078ccedc"
format_stamp: "Formatted at 2026-08-08 22:31:13 on dist-test-slave-7f01"
server_key: "c4655618e8e6b7aeb68a043aafc4933d"
server_key_iv: "b2443cdb2d4e4f053da99715f79611ba"
server_key_version: "encryptionkey@0"
I20260808 22:31:13.097185 8509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "439d32e08064471da6d702da078ccedc"
format_stamp: "Formatted at 2026-08-08 22:31:13 on dist-test-slave-7f01"
server_key: "c4655618e8e6b7aeb68a043aafc4933d"
server_key_iv: "b2443cdb2d4e4f053da99715f79611ba"
server_key_version: "encryptionkey@0"
I20260808 22:31:13.101015 8509 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260808 22:31:13.103372 8537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:13.104661 8509 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 22:31:13.104805 8509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "439d32e08064471da6d702da078ccedc"
format_stamp: "Formatted at 2026-08-08 22:31:13 on dist-test-slave-7f01"
server_key: "c4655618e8e6b7aeb68a043aafc4933d"
server_key_iv: "b2443cdb2d4e4f053da99715f79611ba"
server_key_version: "encryptionkey@0"
I20260808 22:31:13.104918 8509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:13.119349 8509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:13.122117 8509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:13.122321 8509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:13.129614 8509 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:45135
I20260808 22:31:13.129689 8632 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:45135 every 8 connection(s)
I20260808 22:31:13.130596 8509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:31:13.133513 8633 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:13.139001 8633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 439d32e08064471da6d702da078ccedc: Bootstrap starting.
I20260808 22:31:13.140830 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 8509
I20260808 22:31:13.141006 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:31:13.141283 5167 external_mini_cluster.cc:1428] Setting key ee4f7c32c2cc9d849ca02e1085eeb917
I20260808 22:31:13.141614 8633 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 439d32e08064471da6d702da078ccedc: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:13.142478 8633 log.cc:824] T 00000000000000000000000000000000 P 439d32e08064471da6d702da078ccedc: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:13.144687 8633 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 439d32e08064471da6d702da078ccedc: No bootstrap required, opened a new log
I20260808 22:31:13.148205 8633 raft_consensus.cc:348] T 00000000000000000000000000000000 P 439d32e08064471da6d702da078ccedc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "439d32e08064471da6d702da078ccedc" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 45135 } }
I20260808 22:31:13.148460 8633 raft_consensus.cc:374] T 00000000000000000000000000000000 P 439d32e08064471da6d702da078ccedc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:13.148552 8633 raft_consensus.cc:729] T 00000000000000000000000000000000 P 439d32e08064471da6d702da078ccedc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 439d32e08064471da6d702da078ccedc, State: Initialized, Role: FOLLOWER
I20260808 22:31:13.149037 8633 consensus_queue.cc:260] T 00000000000000000000000000000000 P 439d32e08064471da6d702da078ccedc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "439d32e08064471da6d702da078ccedc" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 45135 } }
I20260808 22:31:13.149194 8633 raft_consensus.cc:388] T 00000000000000000000000000000000 P 439d32e08064471da6d702da078ccedc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:31:13.149273 8633 raft_consensus.cc:482] T 00000000000000000000000000000000 P 439d32e08064471da6d702da078ccedc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:31:13.149405 8633 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 439d32e08064471da6d702da078ccedc [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:13.150151 8636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:13.142443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52225 (local address 127.5.11.254:45135)
0808 22:31:13.142906 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:13.142928 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:13.142972 (+ 44us) server_negotiation.cc:407] Connection header received
0808 22:31:13.143717 (+ 745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:13.143778 (+ 61us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:13.144191 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:13.144530 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:13.144956 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:13.145905 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:13.146795 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:13.147273 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:13.147440 (+ 167us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:13.148367 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:13.148372 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:13.148384 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:31:13.148862 (+ 478us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:13.148918 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:31:13.148941 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:13.149083 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:13.149270 (+ 187us) server_negotiation.cc:299] Negotiation successful
0808 22:31:13.149408 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":127,"threads_started":1}
I20260808 22:31:13.150317 8633 raft_consensus.cc:504] T 00000000000000000000000000000000 P 439d32e08064471da6d702da078ccedc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "439d32e08064471da6d702da078ccedc" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 45135 } }
I20260808 22:31:13.150686 8633 leader_election.cc:302] T 00000000000000000000000000000000 P 439d32e08064471da6d702da078ccedc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 439d32e08064471da6d702da078ccedc; no voters:
I20260808 22:31:13.151043 8633 leader_election.cc:288] T 00000000000000000000000000000000 P 439d32e08064471da6d702da078ccedc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:31:13.151140 8640 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 439d32e08064471da6d702da078ccedc [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:13.151437 8640 raft_consensus.cc:686] T 00000000000000000000000000000000 P 439d32e08064471da6d702da078ccedc [term 1 LEADER]: Becoming Leader. State: Replica: 439d32e08064471da6d702da078ccedc, State: Running, Role: LEADER
I20260808 22:31:13.151814 8640 consensus_queue.cc:237] T 00000000000000000000000000000000 P 439d32e08064471da6d702da078ccedc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "439d32e08064471da6d702da078ccedc" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 45135 } }
I20260808 22:31:13.152354 8633 sys_catalog.cc:564] T 00000000000000000000000000000000 P 439d32e08064471da6d702da078ccedc [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:31:13.153009 8640 sys_catalog.cc:455] T 00000000000000000000000000000000 P 439d32e08064471da6d702da078ccedc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 439d32e08064471da6d702da078ccedc. Latest consensus state: current_term: 1 leader_uuid: "439d32e08064471da6d702da078ccedc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "439d32e08064471da6d702da078ccedc" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 45135 } } }
I20260808 22:31:13.153188 8640 sys_catalog.cc:458] T 00000000000000000000000000000000 P 439d32e08064471da6d702da078ccedc [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:13.153347 5167 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:31:13.153479 8641 sys_catalog.cc:455] T 00000000000000000000000000000000 P 439d32e08064471da6d702da078ccedc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "439d32e08064471da6d702da078ccedc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "439d32e08064471da6d702da078ccedc" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 45135 } } }
I20260808 22:31:13.153575 8641 sys_catalog.cc:458] T 00000000000000000000000000000000 P 439d32e08064471da6d702da078ccedc [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:13.153627 8654 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:31:13.156944 8654 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:31:13.160652 8636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:13.155566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54549 (local address 127.5.11.254:45135)
0808 22:31:13.155687 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:13.155691 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:13.156078 (+ 387us) server_negotiation.cc:407] Connection header received
0808 22:31:13.156260 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:13.156263 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:13.156304 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:13.156353 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:13.156743 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:13.157482 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:13.158599 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:13.158935 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:13.159022 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:13.160154 (+ 1132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:13.160157 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:13.160160 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:31:13.160172 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:13.160199 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:31:13.160211 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:13.160268 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:13.160499 (+ 231us) server_negotiation.cc:299] Negotiation successful
0808 22:31:13.160534 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260808 22:31:13.163542 8654 catalog_manager.cc:1347] Generated new cluster ID: 302edace65464dfd90b99f9d32f10d91
I20260808 22:31:13.163604 8654 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:31:13.171943 8654 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:31:13.173103 8654 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:31:13.183559 8654 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 439d32e08064471da6d702da078ccedc: Generated new TSK 0
I20260808 22:31:13.184324 8654 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:31:13.195788 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 8509
W20260808 22:31:13.207579 8486 connection.cc:535] client connection to 127.5.11.254:45135 recv error: Network error: failed to read from TLS socket (remote: 127.5.11.254:45135): Connection reset by peer (error 104)
2026-08-08T22:31:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1634 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:31:13 dist-test-slave-7f01 krb5kdc[8687](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:31:13 dist-test-slave-7f01 krb5kdc[8687](info): set up 2 sockets
Aug 08 22:31:13 dist-test-slave-7f01 krb5kdc[8687](info): commencing operation
krb5kdc: starting...
W20260808 22:31:14.458648 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.237s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:31:14 dist-test-slave-7f01 krb5kdc[8687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228274, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:31:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:31:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.11.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
I20260808 22:31:14.586220 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:36601
--webserver_interface=127.5.11.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:39821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:36601
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:14.698989 8713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:14.699239 8713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:14.699278 8713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:14.703141 8713 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:31:14.703189 8713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:14.703208 8713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:14.703224 8713 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:31:14.703239 8713 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:31:14.708300 8713 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:39821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:36601
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:36601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8713
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:14.709219 8713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:14.710144 8713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:14.718713 8723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:14.718725 8725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:14.718870 8713 server_base.cc:1034] running on GCE node
W20260808 22:31:14.718725 8722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:14.719628 8713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:14.720703 8713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:14.721879 8713 hybrid_clock.cc:648] HybridClock initialized: now 1786228274721850 us; error 47 us; skew 500 ppm
Aug 08 22:31:14 dist-test-slave-7f01 krb5kdc[8687](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228274, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:14.724769 8713 init.cc:377] Logged in from keytab as kudu/127.5.11.254@KRBTEST.COM (short username kudu)
I20260808 22:31:14.725695 8713 webserver.cc:460] Webserver started at http://127.5.11.254:37983/ using document root <none> and password file <none>
I20260808 22:31:14.726280 8713 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:14.726375 8713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:14.726613 8713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:14.728699 8713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "95c34002e6b9416c8db4751c05fa7ebb"
format_stamp: "Formatted at 2026-08-08 22:31:14 on dist-test-slave-7f01"
server_key: "95f5676e84cf9fdf352d797ab02cc25b"
server_key_iv: "ee1334b4072cabca3142ed06649c1825"
server_key_version: "encryptionkey@0"
I20260808 22:31:14.729277 8713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "95c34002e6b9416c8db4751c05fa7ebb"
format_stamp: "Formatted at 2026-08-08 22:31:14 on dist-test-slave-7f01"
server_key: "95f5676e84cf9fdf352d797ab02cc25b"
server_key_iv: "ee1334b4072cabca3142ed06649c1825"
server_key_version: "encryptionkey@0"
I20260808 22:31:14.732934 8713 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260808 22:31:14.735266 8734 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:14.736626 8713 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260808 22:31:14.736788 8713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "95c34002e6b9416c8db4751c05fa7ebb"
format_stamp: "Formatted at 2026-08-08 22:31:14 on dist-test-slave-7f01"
server_key: "95f5676e84cf9fdf352d797ab02cc25b"
server_key_iv: "ee1334b4072cabca3142ed06649c1825"
server_key_version: "encryptionkey@0"
I20260808 22:31:14.736910 8713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:14.741506 8713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:14.744546 8713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:14.744751 8713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:14.752002 8713 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:36601
I20260808 22:31:14.752029 8827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:36601 every 8 connection(s)
I20260808 22:31:14.752908 8713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:31:14.755584 8828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:14.760915 8828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95c34002e6b9416c8db4751c05fa7ebb: Bootstrap starting.
I20260808 22:31:14.762274 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 8713
I20260808 22:31:14.762439 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:31:14.762732 5167 external_mini_cluster.cc:1428] Setting key bfdf4d44aee5b5f51f0753509a06e871
I20260808 22:31:14.763614 8828 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 95c34002e6b9416c8db4751c05fa7ebb: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:14.764532 8828 log.cc:824] T 00000000000000000000000000000000 P 95c34002e6b9416c8db4751c05fa7ebb: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:14.766467 8828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 95c34002e6b9416c8db4751c05fa7ebb: No bootstrap required, opened a new log
I20260808 22:31:14.767598 8832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:14.763863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44943 (local address 127.5.11.254:36601)
0808 22:31:14.764329 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:14.764344 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:14.764384 (+ 40us) server_negotiation.cc:407] Connection header received
0808 22:31:14.765060 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:14.765079 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:14.765339 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:14.765677 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:14.766452 (+ 775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:14.766457 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:14.766468 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:31:14.766478 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:14.766533 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:31:14.766552 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:14.766675 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:14.766801 (+ 126us) server_negotiation.cc:299] Negotiation successful
0808 22:31:14.766958 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":127,"threads_started":1}
I20260808 22:31:14.770383 5167 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:31:14.770638 8828 raft_consensus.cc:348] T 00000000000000000000000000000000 P 95c34002e6b9416c8db4751c05fa7ebb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95c34002e6b9416c8db4751c05fa7ebb" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 36601 } }
I20260808 22:31:14.770870 8828 raft_consensus.cc:374] T 00000000000000000000000000000000 P 95c34002e6b9416c8db4751c05fa7ebb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:14.770928 8828 raft_consensus.cc:729] T 00000000000000000000000000000000 P 95c34002e6b9416c8db4751c05fa7ebb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95c34002e6b9416c8db4751c05fa7ebb, State: Initialized, Role: FOLLOWER
I20260808 22:31:14.771301 8828 consensus_queue.cc:260] T 00000000000000000000000000000000 P 95c34002e6b9416c8db4751c05fa7ebb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95c34002e6b9416c8db4751c05fa7ebb" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 36601 } }
I20260808 22:31:14.771456 8828 raft_consensus.cc:388] T 00000000000000000000000000000000 P 95c34002e6b9416c8db4751c05fa7ebb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:31:14.771548 8828 raft_consensus.cc:482] T 00000000000000000000000000000000 P 95c34002e6b9416c8db4751c05fa7ebb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:31:14.771654 8828 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 95c34002e6b9416c8db4751c05fa7ebb [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:14.772769 8828 raft_consensus.cc:504] T 00000000000000000000000000000000 P 95c34002e6b9416c8db4751c05fa7ebb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95c34002e6b9416c8db4751c05fa7ebb" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 36601 } }
I20260808 22:31:14.773157 8828 leader_election.cc:302] T 00000000000000000000000000000000 P 95c34002e6b9416c8db4751c05fa7ebb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 95c34002e6b9416c8db4751c05fa7ebb; no voters:
I20260808 22:31:14.773478 8828 leader_election.cc:288] T 00000000000000000000000000000000 P 95c34002e6b9416c8db4751c05fa7ebb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:31:14.773581 8856 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 95c34002e6b9416c8db4751c05fa7ebb [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:14.773847 8856 raft_consensus.cc:686] T 00000000000000000000000000000000 P 95c34002e6b9416c8db4751c05fa7ebb [term 1 LEADER]: Becoming Leader. State: Replica: 95c34002e6b9416c8db4751c05fa7ebb, State: Running, Role: LEADER
I20260808 22:31:14.774381 8832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:14.772513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43729 (local address 127.5.11.254:36601)
0808 22:31:14.772620 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:14.772623 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:14.772845 (+ 222us) server_negotiation.cc:407] Connection header received
0808 22:31:14.773046 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:14.773049 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:14.773086 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:14.773169 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:14.773875 (+ 706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:14.773879 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:14.773881 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:31:14.773897 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:14.773924 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:31:14.773939 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:14.774002 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:14.774211 (+ 209us) server_negotiation.cc:299] Negotiation successful
0808 22:31:14.774252 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 22:31:14.774251 8856 consensus_queue.cc:237] T 00000000000000000000000000000000 P 95c34002e6b9416c8db4751c05fa7ebb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95c34002e6b9416c8db4751c05fa7ebb" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 36601 } }
I20260808 22:31:14.774782 8828 sys_catalog.cc:564] T 00000000000000000000000000000000 P 95c34002e6b9416c8db4751c05fa7ebb [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:31:14.775674 8856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95c34002e6b9416c8db4751c05fa7ebb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 95c34002e6b9416c8db4751c05fa7ebb. Latest consensus state: current_term: 1 leader_uuid: "95c34002e6b9416c8db4751c05fa7ebb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95c34002e6b9416c8db4751c05fa7ebb" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 36601 } } }
I20260808 22:31:14.775825 8856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95c34002e6b9416c8db4751c05fa7ebb [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:14.776131 8857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 95c34002e6b9416c8db4751c05fa7ebb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "95c34002e6b9416c8db4751c05fa7ebb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "95c34002e6b9416c8db4751c05fa7ebb" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 36601 } } }
I20260808 22:31:14.776214 8872 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:31:14.776238 8857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 95c34002e6b9416c8db4751c05fa7ebb [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:14.779424 8872 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:31:14.784396 8872 catalog_manager.cc:1347] Generated new cluster ID: bf9f434af187451eadbacd1db2c9ca64
I20260808 22:31:14.784461 8872 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:31:14.803643 8872 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:31:14.803958 8872 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:31:14.823711 8872 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 95c34002e6b9416c8db4751c05fa7ebb: Generated new TSK 0
I20260808 22:31:14.824417 8872 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:31:14.828789 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 8713
2026-08-08T22:31:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1634 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:31:14 dist-test-slave-7f01 krb5kdc[8894](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:31:14 dist-test-slave-7f01 krb5kdc[8894](info): set up 2 sockets
Aug 08 22:31:14 dist-test-slave-7f01 krb5kdc[8894](info): commencing operation
krb5kdc: starting...
W20260808 22:31:16.105486 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.248s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:31:16 dist-test-slave-7f01 krb5kdc[8894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:31:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:31:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.11.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
I20260808 22:31:16.230736 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:38961
--webserver_interface=127.5.11.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:36631
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:38961
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:16.343704 8913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:16.344051 8913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:16.344136 8913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:16.348096 8913 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:31:16.348186 8913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:16.348225 8913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:16.348263 8913 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:31:16.348295 8913 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:31:16.353670 8913 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:36631
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:38961
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:38961
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:16.354743 8913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:16.355826 8913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:16.364356 8924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:16.364355 8925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:16.364528 8913 server_base.cc:1034] running on GCE node
W20260808 22:31:16.364416 8928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:16.365293 8913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:16.366372 8913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:16.367574 8913 hybrid_clock.cc:648] HybridClock initialized: now 1786228276367554 us; error 31 us; skew 500 ppm
Aug 08 22:31:16 dist-test-slave-7f01 krb5kdc[8894](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228276, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:16.375061 8913 init.cc:377] Logged in from keytab as kudu/127.5.11.254@KRBTEST.COM (short username kudu)
I20260808 22:31:16.376139 8913 webserver.cc:460] Webserver started at http://127.5.11.254:33477/ using document root <none> and password file <none>
I20260808 22:31:16.376737 8913 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:16.376832 8913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:16.377055 8913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:16.379122 8913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "b8a0b02832c344bd8f7c6e600f65cb92"
format_stamp: "Formatted at 2026-08-08 22:31:16 on dist-test-slave-7f01"
server_key: "13c33ab3570adc437e3fb896b63098dd"
server_key_iv: "c56caef596fc5752ba6cbf40fd16e385"
server_key_version: "encryptionkey@0"
I20260808 22:31:16.379724 8913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "b8a0b02832c344bd8f7c6e600f65cb92"
format_stamp: "Formatted at 2026-08-08 22:31:16 on dist-test-slave-7f01"
server_key: "13c33ab3570adc437e3fb896b63098dd"
server_key_iv: "c56caef596fc5752ba6cbf40fd16e385"
server_key_version: "encryptionkey@0"
I20260808 22:31:16.383438 8913 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 22:31:16.385915 8937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:16.387253 8913 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:31:16.387408 8913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "b8a0b02832c344bd8f7c6e600f65cb92"
format_stamp: "Formatted at 2026-08-08 22:31:16 on dist-test-slave-7f01"
server_key: "13c33ab3570adc437e3fb896b63098dd"
server_key_iv: "c56caef596fc5752ba6cbf40fd16e385"
server_key_version: "encryptionkey@0"
I20260808 22:31:16.387519 8913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:16.402598 8913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:16.405932 8913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:16.406138 8913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:16.413560 8913 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:38961
I20260808 22:31:16.413574 9041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:38961 every 8 connection(s)
I20260808 22:31:16.414531 8913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:31:16.417071 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 8913
I20260808 22:31:16.417222 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:31:16.417498 5167 external_mini_cluster.cc:1428] Setting key 39e910997d20f669541592bc9c1ab2f7
I20260808 22:31:16.417635 9042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:16.424863 9042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8a0b02832c344bd8f7c6e600f65cb92: Bootstrap starting.
Aug 08 22:31:16 dist-test-slave-7f01 krb5kdc[8894](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228276, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:31:16.427624 9042 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8a0b02832c344bd8f7c6e600f65cb92: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:16.428654 9042 log.cc:824] T 00000000000000000000000000000000 P b8a0b02832c344bd8f7c6e600f65cb92: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:16.430549 9042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8a0b02832c344bd8f7c6e600f65cb92: No bootstrap required, opened a new log
I20260808 22:31:16.432319 9045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:16.418853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51999 (local address 127.5.11.254:38961)
0808 22:31:16.419281 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:16.419296 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:16.419332 (+ 36us) server_negotiation.cc:407] Connection header received
0808 22:31:16.420362 (+ 1030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:16.420382 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:16.420722 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:16.421002 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:16.421968 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:16.422943 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:16.424152 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:16.424614 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:16.424734 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:16.427557 (+ 2823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:16.427562 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:16.427576 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:16.427603 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:16.429418 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:16.429881 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:16.429886 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:16.429897 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:16.430009 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:16.430305 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:16.430309 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:16.430310 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:16.430756 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:16.430972 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:16.431243 (+ 271us) server_negotiation.cc:299] Negotiation successful
0808 22:31:16.431536 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":147,"threads_started":1}
I20260808 22:31:16.434577 9042 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8a0b02832c344bd8f7c6e600f65cb92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8a0b02832c344bd8f7c6e600f65cb92" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 38961 } }
I20260808 22:31:16.434875 9042 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8a0b02832c344bd8f7c6e600f65cb92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:16.434942 9042 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8a0b02832c344bd8f7c6e600f65cb92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8a0b02832c344bd8f7c6e600f65cb92, State: Initialized, Role: FOLLOWER
I20260808 22:31:16.435333 5167 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:31:16.435385 9042 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8a0b02832c344bd8f7c6e600f65cb92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8a0b02832c344bd8f7c6e600f65cb92" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 38961 } }
I20260808 22:31:16.435580 9042 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8a0b02832c344bd8f7c6e600f65cb92 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:31:16.435652 9042 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8a0b02832c344bd8f7c6e600f65cb92 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:31:16.435787 9042 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8a0b02832c344bd8f7c6e600f65cb92 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:16.436719 9042 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8a0b02832c344bd8f7c6e600f65cb92 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8a0b02832c344bd8f7c6e600f65cb92" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 38961 } }
I20260808 22:31:16.437088 9042 leader_election.cc:302] T 00000000000000000000000000000000 P b8a0b02832c344bd8f7c6e600f65cb92 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8a0b02832c344bd8f7c6e600f65cb92; no voters:
I20260808 22:31:16.437443 9042 leader_election.cc:288] T 00000000000000000000000000000000 P b8a0b02832c344bd8f7c6e600f65cb92 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:31:16.437558 9057 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8a0b02832c344bd8f7c6e600f65cb92 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:16.437832 9057 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8a0b02832c344bd8f7c6e600f65cb92 [term 1 LEADER]: Becoming Leader. State: Replica: b8a0b02832c344bd8f7c6e600f65cb92, State: Running, Role: LEADER
I20260808 22:31:16.438181 9057 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8a0b02832c344bd8f7c6e600f65cb92 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8a0b02832c344bd8f7c6e600f65cb92" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 38961 } }
I20260808 22:31:16.438917 9042 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8a0b02832c344bd8f7c6e600f65cb92 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:31:16.441521 9060 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8a0b02832c344bd8f7c6e600f65cb92 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8a0b02832c344bd8f7c6e600f65cb92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8a0b02832c344bd8f7c6e600f65cb92" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 38961 } } }
I20260808 22:31:16.441648 9060 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8a0b02832c344bd8f7c6e600f65cb92 [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:16.441825 9061 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8a0b02832c344bd8f7c6e600f65cb92 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8a0b02832c344bd8f7c6e600f65cb92. Latest consensus state: current_term: 1 leader_uuid: "b8a0b02832c344bd8f7c6e600f65cb92" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8a0b02832c344bd8f7c6e600f65cb92" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 38961 } } }
I20260808 22:31:16.442262 9061 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8a0b02832c344bd8f7c6e600f65cb92 [sys.catalog]: This master's current role is: LEADER
W20260808 22:31:16.444471 9087 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b8a0b02832c344bd8f7c6e600f65cb92: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:31:16.444556 9087 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:31:16.444638 9088 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:31:16.445623 9088 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:31:16.446660 9045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:16.437530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49019 (local address 127.5.11.254:38961)
0808 22:31:16.437791 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:16.437798 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:16.437872 (+ 74us) server_negotiation.cc:407] Connection header received
0808 22:31:16.438049 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:16.438052 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:16.438094 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:16.438145 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:16.439102 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:16.439709 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:16.440775 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:16.441120 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:16.441224 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:16.443038 (+ 1814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:16.443042 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:16.443045 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:16.443069 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:16.444855 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:16.445326 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:16.445329 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:16.445331 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:16.445405 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:16.445729 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:16.445735 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:16.445740 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:16.445972 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:16.446106 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:16.446392 (+ 286us) server_negotiation.cc:299] Negotiation successful
0808 22:31:16.446508 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177}
I20260808 22:31:16.451030 9088 catalog_manager.cc:1347] Generated new cluster ID: 444a397480f943bdaf2059e1e7113b86
I20260808 22:31:16.451100 9088 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:31:16.472663 9088 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:31:16.473631 9088 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:31:16.480324 9088 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8a0b02832c344bd8f7c6e600f65cb92: Generated new TSK 0
I20260808 22:31:16.480999 9088 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:31:16.487558 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 8913
2026-08-08T22:31:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1655 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:31:16 dist-test-slave-7f01 krb5kdc[9104](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:31:16 dist-test-slave-7f01 krb5kdc[9104](info): set up 2 sockets
Aug 08 22:31:16 dist-test-slave-7f01 krb5kdc[9104](info): commencing operation
krb5kdc: starting...
W20260808 22:31:17.749154 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.236s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:31:17 dist-test-slave-7f01 krb5kdc[9104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228277, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:31:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:31:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.11.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
I20260808 22:31:17.879036 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:36443
--webserver_interface=127.5.11.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:33755
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:36443
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:17.996202 9127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:17.996466 9127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:17.996507 9127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:18.000586 9127 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:31:18.000643 9127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:18.000663 9127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:18.000718 9127 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:31:18.000751 9127 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:31:18.006033 9127 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:33755
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:36443
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:36443
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:18.007138 9127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:18.008386 9127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:18.017215 9134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:18.017220 9137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:18.017277 9127 server_base.cc:1034] running on GCE node
W20260808 22:31:18.017227 9135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:18.018163 9127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:18.019266 9127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:18.020470 9127 hybrid_clock.cc:648] HybridClock initialized: now 1786228278020445 us; error 45 us; skew 500 ppm
Aug 08 22:31:18 dist-test-slave-7f01 krb5kdc[9104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228278, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:18.023300 9127 init.cc:377] Logged in from keytab as kudu/127.5.11.254@KRBTEST.COM (short username kudu)
I20260808 22:31:18.024240 9127 webserver.cc:460] Webserver started at http://127.5.11.254:34959/ using document root <none> and password file <none>
I20260808 22:31:18.024811 9127 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:18.024904 9127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:18.025135 9127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:18.027168 9127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "ef63beecf1554ff8b63fcd7ad4cb746b"
format_stamp: "Formatted at 2026-08-08 22:31:18 on dist-test-slave-7f01"
server_key: "49e0a4b67436ca5ad6b15355ff7985d3"
server_key_iv: "023551c7d4d4916a6440db49d79cba29"
server_key_version: "encryptionkey@0"
I20260808 22:31:18.027796 9127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "ef63beecf1554ff8b63fcd7ad4cb746b"
format_stamp: "Formatted at 2026-08-08 22:31:18 on dist-test-slave-7f01"
server_key: "49e0a4b67436ca5ad6b15355ff7985d3"
server_key_iv: "023551c7d4d4916a6440db49d79cba29"
server_key_version: "encryptionkey@0"
I20260808 22:31:18.031353 9127 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260808 22:31:18.033852 9149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:18.035105 9127 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:31:18.035262 9127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "ef63beecf1554ff8b63fcd7ad4cb746b"
format_stamp: "Formatted at 2026-08-08 22:31:18 on dist-test-slave-7f01"
server_key: "49e0a4b67436ca5ad6b15355ff7985d3"
server_key_iv: "023551c7d4d4916a6440db49d79cba29"
server_key_version: "encryptionkey@0"
I20260808 22:31:18.035377 9127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:18.046665 9127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:18.049714 9127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:18.049933 9127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:18.057482 9127 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:36443
I20260808 22:31:18.057546 9243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:36443 every 8 connection(s)
I20260808 22:31:18.058492 9127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:31:18.061393 9244 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:18.066955 9244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef63beecf1554ff8b63fcd7ad4cb746b: Bootstrap starting.
I20260808 22:31:18.067432 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 9127
I20260808 22:31:18.067576 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:31:18.067898 5167 external_mini_cluster.cc:1428] Setting key 63ca8e9c5e1ce070fc9b797fd553aff9
I20260808 22:31:18.069840 9244 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ef63beecf1554ff8b63fcd7ad4cb746b: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:18.070570 9244 log.cc:824] T 00000000000000000000000000000000 P ef63beecf1554ff8b63fcd7ad4cb746b: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:18.072724 9244 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ef63beecf1554ff8b63fcd7ad4cb746b: No bootstrap required, opened a new log
I20260808 22:31:18.076077 9244 raft_consensus.cc:348] T 00000000000000000000000000000000 P ef63beecf1554ff8b63fcd7ad4cb746b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef63beecf1554ff8b63fcd7ad4cb746b" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 36443 } }
I20260808 22:31:18.076385 9244 raft_consensus.cc:374] T 00000000000000000000000000000000 P ef63beecf1554ff8b63fcd7ad4cb746b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:18.076470 9244 raft_consensus.cc:729] T 00000000000000000000000000000000 P ef63beecf1554ff8b63fcd7ad4cb746b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef63beecf1554ff8b63fcd7ad4cb746b, State: Initialized, Role: FOLLOWER
I20260808 22:31:18.076951 9244 consensus_queue.cc:260] T 00000000000000000000000000000000 P ef63beecf1554ff8b63fcd7ad4cb746b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef63beecf1554ff8b63fcd7ad4cb746b" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 36443 } }
I20260808 22:31:18.077077 9244 raft_consensus.cc:388] T 00000000000000000000000000000000 P ef63beecf1554ff8b63fcd7ad4cb746b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:31:18.077128 9248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:18.069314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42473 (local address 127.5.11.254:36443)
0808 22:31:18.069794 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:18.069809 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:18.069846 (+ 37us) server_negotiation.cc:407] Connection header received
0808 22:31:18.070516 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:18.070535 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:18.070979 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:18.071364 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:18.071720 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:18.072743 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:18.073759 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:18.074244 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:18.074419 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:18.075296 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:18.075301 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:18.075313 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:31:18.075687 (+ 374us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:18.075797 (+ 110us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:31:18.075835 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:18.075941 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:18.076068 (+ 127us) server_negotiation.cc:299] Negotiation successful
0808 22:31:18.076276 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":144,"threads_started":1}
I20260808 22:31:18.077159 9244 raft_consensus.cc:482] T 00000000000000000000000000000000 P ef63beecf1554ff8b63fcd7ad4cb746b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:31:18.077275 9244 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ef63beecf1554ff8b63fcd7ad4cb746b [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:18.078202 9244 raft_consensus.cc:504] T 00000000000000000000000000000000 P ef63beecf1554ff8b63fcd7ad4cb746b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef63beecf1554ff8b63fcd7ad4cb746b" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 36443 } }
I20260808 22:31:18.078577 9244 leader_election.cc:302] T 00000000000000000000000000000000 P ef63beecf1554ff8b63fcd7ad4cb746b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ef63beecf1554ff8b63fcd7ad4cb746b; no voters:
I20260808 22:31:18.078905 9244 leader_election.cc:288] T 00000000000000000000000000000000 P ef63beecf1554ff8b63fcd7ad4cb746b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:31:18.079013 9250 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ef63beecf1554ff8b63fcd7ad4cb746b [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:18.079248 9250 raft_consensus.cc:686] T 00000000000000000000000000000000 P ef63beecf1554ff8b63fcd7ad4cb746b [term 1 LEADER]: Becoming Leader. State: Replica: ef63beecf1554ff8b63fcd7ad4cb746b, State: Running, Role: LEADER
I20260808 22:31:18.079639 9250 consensus_queue.cc:237] T 00000000000000000000000000000000 P ef63beecf1554ff8b63fcd7ad4cb746b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef63beecf1554ff8b63fcd7ad4cb746b" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 36443 } }
I20260808 22:31:18.080233 9244 sys_catalog.cc:564] T 00000000000000000000000000000000 P ef63beecf1554ff8b63fcd7ad4cb746b [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:31:18.080367 5167 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:31:18.081521 9251 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef63beecf1554ff8b63fcd7ad4cb746b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ef63beecf1554ff8b63fcd7ad4cb746b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef63beecf1554ff8b63fcd7ad4cb746b" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 36443 } } }
I20260808 22:31:18.081650 9251 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef63beecf1554ff8b63fcd7ad4cb746b [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:18.082068 9252 sys_catalog.cc:455] T 00000000000000000000000000000000 P ef63beecf1554ff8b63fcd7ad4cb746b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ef63beecf1554ff8b63fcd7ad4cb746b. Latest consensus state: current_term: 1 leader_uuid: "ef63beecf1554ff8b63fcd7ad4cb746b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ef63beecf1554ff8b63fcd7ad4cb746b" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 36443 } } }
I20260808 22:31:18.082778 9252 sys_catalog.cc:458] T 00000000000000000000000000000000 P ef63beecf1554ff8b63fcd7ad4cb746b [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:18.083256 9271 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:31:18.086339 9271 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:31:18.087409 9248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:18.082904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33867 (local address 127.5.11.254:36443)
0808 22:31:18.083010 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:18.083015 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:18.083151 (+ 136us) server_negotiation.cc:407] Connection header received
0808 22:31:18.083328 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:18.083332 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:18.083384 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:18.083468 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:18.083801 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:18.084531 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:18.085669 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:18.085987 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:18.086099 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:18.086936 (+ 837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:18.086940 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:18.086943 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:31:18.086955 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:18.086985 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:31:18.086999 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:18.087081 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:18.087225 (+ 144us) server_negotiation.cc:299] Negotiation successful
0808 22:31:18.087287 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 22:31:18.092962 9271 catalog_manager.cc:1347] Generated new cluster ID: d80739f519614a5d8cc33a9fedc75922
I20260808 22:31:18.093043 9271 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:31:18.103586 9271 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:31:18.104549 9271 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:31:18.115401 9271 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ef63beecf1554ff8b63fcd7ad4cb746b: Generated new TSK 0
I20260808 22:31:18.116134 9271 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:31:18.121044 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 9127
2026-08-08T22:31:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1648 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:31:18 dist-test-slave-7f01 krb5kdc[9298](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:31:18 dist-test-slave-7f01 krb5kdc[9298](info): set up 2 sockets
Aug 08 22:31:18 dist-test-slave-7f01 krb5kdc[9298](info): commencing operation
krb5kdc: starting...
W20260808 22:31:19.394294 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:31:19 dist-test-slave-7f01 krb5kdc[9298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228279, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:31:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:31:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.11.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
I20260808 22:31:19.523517 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:39657
--webserver_interface=127.5.11.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:36207
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:39657
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:19.636770 9322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:19.637133 9322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:19.637238 9322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:19.641204 9322 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:31:19.641299 9322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:19.641340 9322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:19.641398 9322 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:31:19.641433 9322 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:31:19.646489 9322 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:36207
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:39657
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:39657
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:19.647516 9322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:19.648635 9322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:19.657080 9333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:19.657086 9335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:19.657193 9322 server_base.cc:1034] running on GCE node
W20260808 22:31:19.657086 9338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:19.657960 9322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:19.659142 9322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:19.660334 9322 hybrid_clock.cc:648] HybridClock initialized: now 1786228279660314 us; error 33 us; skew 500 ppm
Aug 08 22:31:19 dist-test-slave-7f01 krb5kdc[9298](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228279, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:19.663096 9322 init.cc:377] Logged in from keytab as kudu/127.5.11.254@KRBTEST.COM (short username kudu)
I20260808 22:31:19.664156 9322 webserver.cc:460] Webserver started at http://127.5.11.254:44099/ using document root <none> and password file <none>
I20260808 22:31:19.664747 9322 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:19.664844 9322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:19.665087 9322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:19.667193 9322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "8af2bd1eee2446c9ae6d22f986f26af0"
format_stamp: "Formatted at 2026-08-08 22:31:19 on dist-test-slave-7f01"
server_key: "d133a7c0fda0038f865fd2bc5184613e"
server_key_iv: "85ad85a9d25618818e9ddc442897f276"
server_key_version: "encryptionkey@0"
I20260808 22:31:19.667856 9322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "8af2bd1eee2446c9ae6d22f986f26af0"
format_stamp: "Formatted at 2026-08-08 22:31:19 on dist-test-slave-7f01"
server_key: "d133a7c0fda0038f865fd2bc5184613e"
server_key_iv: "85ad85a9d25618818e9ddc442897f276"
server_key_version: "encryptionkey@0"
I20260808 22:31:19.671543 9322 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:31:19.674002 9351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:19.675227 9322 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:31:19.675387 9322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "8af2bd1eee2446c9ae6d22f986f26af0"
format_stamp: "Formatted at 2026-08-08 22:31:19 on dist-test-slave-7f01"
server_key: "d133a7c0fda0038f865fd2bc5184613e"
server_key_iv: "85ad85a9d25618818e9ddc442897f276"
server_key_version: "encryptionkey@0"
I20260808 22:31:19.675504 9322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:19.680186 9322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:19.683238 9322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:19.683451 9322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:19.691077 9322 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:39657
I20260808 22:31:19.691107 9438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:39657 every 8 connection(s)
I20260808 22:31:19.692106 9322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:31:19.694885 9439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:19.700086 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 9322
I20260808 22:31:19.700243 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:31:19.700531 5167 external_mini_cluster.cc:1428] Setting key fb198dead78a29a5ac75f8967bae4b14
I20260808 22:31:19.700549 9439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8af2bd1eee2446c9ae6d22f986f26af0: Bootstrap starting.
I20260808 22:31:19.702999 9439 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8af2bd1eee2446c9ae6d22f986f26af0: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:19.703784 9439 log.cc:824] T 00000000000000000000000000000000 P 8af2bd1eee2446c9ae6d22f986f26af0: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:19.705802 9444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:19.701884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58543 (local address 127.5.11.254:39657)
0808 22:31:19.702321 (+ 437us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:19.702336 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:19.702370 (+ 34us) server_negotiation.cc:407] Connection header received
0808 22:31:19.702998 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:19.703015 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:19.703340 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:19.703694 (+ 354us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:19.704506 (+ 812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:19.704513 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:19.704530 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:31:19.704543 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:19.704608 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:31:19.704641 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:19.704758 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:19.704927 (+ 169us) server_negotiation.cc:299] Negotiation successful
0808 22:31:19.705110 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":120,"threads_started":1}
I20260808 22:31:19.706899 9439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8af2bd1eee2446c9ae6d22f986f26af0: No bootstrap required, opened a new log
I20260808 22:31:19.708630 5167 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:31:19.711498 9439 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8af2bd1eee2446c9ae6d22f986f26af0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8af2bd1eee2446c9ae6d22f986f26af0" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39657 } }
I20260808 22:31:19.711802 9439 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8af2bd1eee2446c9ae6d22f986f26af0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260808 22:31:19.711865 9439 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8af2bd1eee2446c9ae6d22f986f26af0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8af2bd1eee2446c9ae6d22f986f26af0, State: Initialized, Role: FOLLOWER
I20260808 22:31:19.712328 9439 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8af2bd1eee2446c9ae6d22f986f26af0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8af2bd1eee2446c9ae6d22f986f26af0" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39657 } }
I20260808 22:31:19.712484 9439 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8af2bd1eee2446c9ae6d22f986f26af0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260808 22:31:19.712560 9439 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8af2bd1eee2446c9ae6d22f986f26af0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260808 22:31:19.712707 9444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:19.710975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53501 (local address 127.5.11.254:39657)
0808 22:31:19.711103 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:19.711108 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:19.711274 (+ 166us) server_negotiation.cc:407] Connection header received
0808 22:31:19.711392 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:19.711394 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:19.711419 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:19.711483 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:19.712297 (+ 814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:19.712301 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:19.712303 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0808 22:31:19.712314 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:19.712346 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0808 22:31:19.712357 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:19.712444 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:19.712545 (+ 101us) server_negotiation.cc:299] Negotiation successful
0808 22:31:19.712591 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260808 22:31:19.712720 9439 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8af2bd1eee2446c9ae6d22f986f26af0 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:19.713765 9439 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8af2bd1eee2446c9ae6d22f986f26af0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8af2bd1eee2446c9ae6d22f986f26af0" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39657 } }
I20260808 22:31:19.714062 9439 leader_election.cc:302] T 00000000000000000000000000000000 P 8af2bd1eee2446c9ae6d22f986f26af0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8af2bd1eee2446c9ae6d22f986f26af0; no voters:
I20260808 22:31:19.714354 9439 leader_election.cc:288] T 00000000000000000000000000000000 P 8af2bd1eee2446c9ae6d22f986f26af0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260808 22:31:19.714461 9453 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8af2bd1eee2446c9ae6d22f986f26af0 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:19.714720 9453 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8af2bd1eee2446c9ae6d22f986f26af0 [term 1 LEADER]: Becoming Leader. State: Replica: 8af2bd1eee2446c9ae6d22f986f26af0, State: Running, Role: LEADER
I20260808 22:31:19.715083 9453 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8af2bd1eee2446c9ae6d22f986f26af0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8af2bd1eee2446c9ae6d22f986f26af0" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39657 } }
I20260808 22:31:19.715656 9439 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8af2bd1eee2446c9ae6d22f986f26af0 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:31:19.716311 9455 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8af2bd1eee2446c9ae6d22f986f26af0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8af2bd1eee2446c9ae6d22f986f26af0. Latest consensus state: current_term: 1 leader_uuid: "8af2bd1eee2446c9ae6d22f986f26af0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8af2bd1eee2446c9ae6d22f986f26af0" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39657 } } }
I20260808 22:31:19.716434 9455 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8af2bd1eee2446c9ae6d22f986f26af0 [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:19.716797 9454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8af2bd1eee2446c9ae6d22f986f26af0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8af2bd1eee2446c9ae6d22f986f26af0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8af2bd1eee2446c9ae6d22f986f26af0" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 39657 } } }
I20260808 22:31:19.716810 9464 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:31:19.716914 9454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8af2bd1eee2446c9ae6d22f986f26af0 [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:19.720052 9464 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:31:19.725430 9464 catalog_manager.cc:1347] Generated new cluster ID: fe7bf6e01d7f43dd89bd95fb6091dc1e
I20260808 22:31:19.725505 9464 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:31:19.744174 9464 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:31:19.744506 9464 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:31:19.760185 9464 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8af2bd1eee2446c9ae6d22f986f26af0: Generated new TSK 0
I20260808 22:31:19.760926 9464 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260808 22:31:19.764937 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 9322
2026-08-08T22:31:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1630 ms)
[----------] 18 tests from AuthTokenIssuingTest (29538 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:31:19 dist-test-slave-7f01 krb5kdc[9495](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:31:19 dist-test-slave-7f01 krb5kdc[9495](info): set up 2 sockets
Aug 08 22:31:19 dist-test-slave-7f01 krb5kdc[9495](info): commencing operation
krb5kdc: starting...
W20260808 22:31:21.047549 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.256s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:31:21 dist-test-slave-7f01 krb5kdc[9495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:31:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:31:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.11.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
I20260808 22:31:21.186687 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:34561
--webserver_interface=127.5.11.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:45963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:34561,127.5.11.253:39887,127.5.11.252:41809
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:21.302606 9524 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 22:31:21.302989 9524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:21.303068 9524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:21.303146 9524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:21.307315 9524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:31:21.307412 9524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:21.307451 9524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:21.307503 9524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:31:21.307541 9524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:31:21.312873 9524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:45963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:34561,127.5.11.253:39887,127.5.11.252:41809
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:34561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:21.313978 9524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:21.315043 9524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:21.323678 9531 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:21.323694 9532 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:21.323877 9524 server_base.cc:1034] running on GCE node
W20260808 22:31:21.324005 9536 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:21.324662 9524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:21.325737 9524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:21.326941 9524 hybrid_clock.cc:648] HybridClock initialized: now 1786228281326914 us; error 45 us; skew 500 ppm
Aug 08 22:31:21 dist-test-slave-7f01 krb5kdc[9495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228281, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:21.329941 9524 init.cc:377] Logged in from keytab as kudu/127.5.11.254@KRBTEST.COM (short username kudu)
I20260808 22:31:21.330883 9524 webserver.cc:460] Webserver started at http://127.5.11.254:32807/ using document root <none> and password file <none>
I20260808 22:31:21.331450 9524 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:21.331545 9524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:21.331856 9524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:21.333897 9524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "fa6c968612454a12871737d4c2b5ffce"
format_stamp: "Formatted at 2026-08-08 22:31:21 on dist-test-slave-7f01"
server_key: "1f7edd8d87832b907657e37f7a51613a"
server_key_iv: "0e19498e75f9499431834c6b76dcdfc4"
server_key_version: "encryptionkey@0"
I20260808 22:31:21.334484 9524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "fa6c968612454a12871737d4c2b5ffce"
format_stamp: "Formatted at 2026-08-08 22:31:21 on dist-test-slave-7f01"
server_key: "1f7edd8d87832b907657e37f7a51613a"
server_key_iv: "0e19498e75f9499431834c6b76dcdfc4"
server_key_version: "encryptionkey@0"
I20260808 22:31:21.338306 9524 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 22:31:21.340700 9549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:21.341871 9524 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:31:21.342015 9524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "fa6c968612454a12871737d4c2b5ffce"
format_stamp: "Formatted at 2026-08-08 22:31:21 on dist-test-slave-7f01"
server_key: "1f7edd8d87832b907657e37f7a51613a"
server_key_iv: "0e19498e75f9499431834c6b76dcdfc4"
server_key_version: "encryptionkey@0"
I20260808 22:31:21.342127 9524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:21.364539 9524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:21.367841 9524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:21.368046 9524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:21.375133 9524 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:34561
I20260808 22:31:21.375409 9644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:34561 every 8 connection(s)
I20260808 22:31:21.376199 9524 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:31:21.378944 9645 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:21.382681 9645 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.11.254" port: 34561 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:31:21.384042 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 9524
I20260808 22:31:21.384232 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:31:21.384523 5167 external_mini_cluster.cc:1428] Setting key 3554f7a7ada901ba5c7dc955507b4b10
WARNING: no policy specified for kudu/127.5.11.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.253@KRBTEST.COM" created.
Aug 08 22:31:21 dist-test-slave-7f01 krb5kdc[9495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228281, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:31:21.403412 9647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.385076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:34561 (local address 127.0.0.1:48254)
0808 22:31:21.385587 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:21.385641 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:21.386819 (+ 1178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:21.387563 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:21.387570 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:21.388097 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:21.388474 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:21.388491 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.390284 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.390287 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:21.391064 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:21.391074 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.392115 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.392118 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:21.392204 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:21.396628 (+ 4424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:21.396645 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:21.398428 (+ 1783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:21.400418 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:21.400429 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:21.400438 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:21.400667 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:21.400993 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:21.400998 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:21.401000 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:21.401117 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:21.401693 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:21.401699 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:21.401845 (+ 146us) client_negotiation.cc:769] Sending connection context
0808 22:31:21.402018 (+ 173us) client_negotiation.cc:240] Negotiation successful
0808 22:31:21.402418 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":145,"spinlock_wait_cycles":4352,"thread_start_us":153,"threads_started":1}
I20260808 22:31:21.403412 9648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.385981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48254 (local address 127.5.11.254:34561)
0808 22:31:21.386346 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:21.386360 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:21.386406 (+ 46us) server_negotiation.cc:407] Connection header received
0808 22:31:21.387162 (+ 756us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:21.387166 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:21.387343 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:21.387501 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:21.388642 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.390123 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.391199 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.391916 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.392166 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:21.398585 (+ 6419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:21.398592 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:21.398602 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:21.398622 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:21.400301 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:21.400780 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:21.400792 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:21.400794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:21.400899 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:21.401208 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:21.401213 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:21.401215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:21.401579 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:21.401721 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:21.402089 (+ 368us) server_negotiation.cc:299] Negotiation successful
0808 22:31:21.402368 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":278,"spinlock_wait_cycles":4352,"thread_start_us":89,"threads_started":1}
I20260808 22:31:21.405810 9645 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.11.253" port: 39887 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:31:21.406692 9647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.406377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.253:39887 (local address 127.0.0.1:40876)
0808 22:31:21.406498 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:21.406579 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.253:39887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260808 22:31:21.407028 9553 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.11.253:39887: connect: Connection refused (error 111)
I20260808 22:31:21.407986 9647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.407774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.253:39887 (local address 127.0.0.1:40880)
0808 22:31:21.407885 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:21.407933 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.253:39887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260808 22:31:21.408149 9645 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.11.253:39887: Network error: Client connection negotiation failed: client connection to 127.5.11.253:39887: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.253.keytab.
Entry for principal kudu/127.5.11.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.253.keytab.
I20260808 22:31:21.412999 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.253:39887
--webserver_interface=127.5.11.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.253.keytab
--principal=kudu/127.5.11.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:45963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:34561,127.5.11.253:39887,127.5.11.252:41809
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:31:21.457410 9645 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.11.253" port: 39887 } attempt: 1
I20260808 22:31:21.458387 9647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.458058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.253:39887 (local address 127.0.0.1:40894)
0808 22:31:21.458224 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:21.458304 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.253:39887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 22:31:21.459208 9647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.458995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.253:39887 (local address 127.0.0.1:40900)
0808 22:31:21.459098 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:21.459146 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.253:39887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260808 22:31:21.459435 9645 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.11.253:39887: Network error: Client connection negotiation failed: client connection to 127.5.11.253:39887: connect: Connection refused (error 111)
W20260808 22:31:21.526441 9654 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 22:31:21.526716 9654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:21.526788 9654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:21.526867 9654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 22:31:21.527616 9645 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.11.253" port: 39887 } attempt: 2
I20260808 22:31:21.528597 9647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.528225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.253:39887 (local address 127.0.0.1:40904)
0808 22:31:21.528440 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:21.528517 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.253:39887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260808 22:31:21.529336 9647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.529116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.253:39887 (local address 127.0.0.1:40916)
0808 22:31:21.529228 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:21.529281 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.253:39887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 22:31:21.529470 9645 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.11.253:39887: Network error: Client connection negotiation failed: client connection to 127.5.11.253:39887: connect: Connection refused (error 111)
W20260808 22:31:21.531369 9654 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:31:21.531447 9654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:21.531494 9654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:21.531535 9654 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:31:21.531570 9654 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:31:21.536895 9654 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:45963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:34561,127.5.11.253:39887,127.5.11.252:41809
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.253.keytab
--principal=kudu/127.5.11.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.253:39887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.11.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:21.537981 9654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:21.539026 9654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260808 22:31:21.547655 9654 server_base.cc:1034] running on GCE node
W20260808 22:31:21.547919 9662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:21.547886 9665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:21.547958 9660 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:21.548722 9654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:21.549800 9654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:21.550995 9654 hybrid_clock.cc:648] HybridClock initialized: now 1786228281550972 us; error 49 us; skew 500 ppm
Aug 08 22:31:21 dist-test-slave-7f01 krb5kdc[9495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228281, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:21.553913 9654 init.cc:377] Logged in from keytab as kudu/127.5.11.253@KRBTEST.COM (short username kudu)
I20260808 22:31:21.554785 9654 webserver.cc:460] Webserver started at http://127.5.11.253:38721/ using document root <none> and password file <none>
I20260808 22:31:21.555356 9654 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:21.555447 9654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:21.555681 9654 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:21.557761 9654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-1/data/instance:
uuid: "d527a8616eb646d98270bced5643ae1f"
format_stamp: "Formatted at 2026-08-08 22:31:21 on dist-test-slave-7f01"
server_key: "3857219ec893a1cd2f138a7413b1f592"
server_key_iv: "679d1f0b82ec57635f1dace683085c5c"
server_key_version: "encryptionkey@0"
I20260808 22:31:21.558326 9654 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-1/wal/instance:
uuid: "d527a8616eb646d98270bced5643ae1f"
format_stamp: "Formatted at 2026-08-08 22:31:21 on dist-test-slave-7f01"
server_key: "3857219ec893a1cd2f138a7413b1f592"
server_key_iv: "679d1f0b82ec57635f1dace683085c5c"
server_key_version: "encryptionkey@0"
I20260808 22:31:21.562184 9654 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260808 22:31:21.564743 9682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:21.565959 9654 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:31:21.566100 9654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-1/wal
uuid: "d527a8616eb646d98270bced5643ae1f"
format_stamp: "Formatted at 2026-08-08 22:31:21 on dist-test-slave-7f01"
server_key: "3857219ec893a1cd2f138a7413b1f592"
server_key_iv: "679d1f0b82ec57635f1dace683085c5c"
server_key_version: "encryptionkey@0"
I20260808 22:31:21.566211 9654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:21.591945 9654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:21.595317 9654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:21.595530 9654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:21.602543 9654 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.253:39887
I20260808 22:31:21.602820 9772 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.253:39887 every 8 connection(s)
I20260808 22:31:21.603539 9654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-1/data/info.pb
I20260808 22:31:21.606369 9775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:21.609699 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 9654
I20260808 22:31:21.609833 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-1/wal/instance
I20260808 22:31:21.610112 5167 external_mini_cluster.cc:1428] Setting key 127d0bb4e2b98be70539a05e399bdfb8
I20260808 22:31:21.610283 9775 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.11.254" port: 34561 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.5.11.252@KRBTEST.COM; defaulting to no policy
I20260808 22:31:21.620625 9645 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.11.253" port: 39887 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.252@KRBTEST.COM" created.
Aug 08 22:31:21 dist-test-slave-7f01 krb5kdc[9495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228281, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.253@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
Aug 08 22:31:21 dist-test-slave-7f01 krb5kdc[9495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228281, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for kudu/127.5.11.253@KRBTEST.COM
I20260808 22:31:21.630784 9778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.612628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:34561 (local address 127.0.0.1:48258)
0808 22:31:21.613147 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:21.613206 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:21.614474 (+ 1268us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:21.614992 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:21.615004 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:21.615576 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:21.616325 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:21.616346 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.617168 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.617174 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:21.618662 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:21.618676 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.619171 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.619176 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:21.619418 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:21.620067 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:21.620093 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:21.624741 (+ 4648us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:21.627773 (+ 3032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:21.627783 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:21.627793 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:21.628159 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:21.628462 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:21.628466 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:21.628468 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:21.628615 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:21.629065 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:21.629072 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:21.629350 (+ 278us) client_negotiation.cc:769] Sending connection context
0808 22:31:21.629559 (+ 209us) client_negotiation.cc:240] Negotiation successful
0808 22:31:21.629854 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":114,"threads_started":1}
I20260808 22:31:21.631206 9647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.621120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.253:39887 (local address 127.0.0.1:40922)
0808 22:31:21.621249 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:21.621265 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:21.621342 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:21.621939 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:21.621942 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:21.622138 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:21.622229 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:21.622234 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.623156 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.623160 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:21.623738 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:21.623751 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.624561 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.624565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:21.624689 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:21.625190 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:21.625209 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:21.626710 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:21.629353 (+ 2643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:21.629356 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:21.629358 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:21.629611 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:21.629971 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:21.629974 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:21.629976 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:21.630068 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:21.630682 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:21.630685 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:21.630784 (+ 99us) client_negotiation.cc:769] Sending connection context
0808 22:31:21.630902 (+ 118us) client_negotiation.cc:240] Negotiation successful
0808 22:31:21.631034 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 22:31:21.631709 9648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.612223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48258 (local address 127.5.11.254:34561)
0808 22:31:21.612395 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:21.612400 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:21.613330 (+ 930us) server_negotiation.cc:407] Connection header received
0808 22:31:21.614686 (+ 1356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:21.614689 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:21.614736 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:21.614808 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:21.616494 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.617032 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.618783 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.619011 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.619087 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:21.625245 (+ 6158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:21.625252 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:21.625258 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:21.625294 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:21.627637 (+ 2343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:21.628284 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:21.628287 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:21.628288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:21.628361 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:21.628730 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:21.628733 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:21.628735 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:21.628956 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:21.629064 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:21.631512 (+ 2448us) server_negotiation.cc:299] Negotiation successful
0808 22:31:21.631619 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260808 22:31:21.631702 9781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.621237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40922 (local address 127.5.11.253:39887)
0808 22:31:21.621545 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:21.621554 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:21.621573 (+ 19us) server_negotiation.cc:407] Connection header received
0808 22:31:21.621604 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:21.621608 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:21.621793 (+ 185us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:21.621954 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:21.622344 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.623035 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.623890 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.624423 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.624548 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:21.626899 (+ 2351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:21.626907 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:21.626922 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:21.626955 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:21.629221 (+ 2266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:21.629735 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:21.629745 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:21.629750 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:21.629853 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:21.630176 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:21.630182 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:21.630187 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:21.630550 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:21.630709 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:21.631343 (+ 634us) server_negotiation.cc:299] Negotiation successful
0808 22:31:21.631500 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":118,"threads_started":1}
I20260808 22:31:21.632889 9775 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.11.253" port: 39887 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:31:21.633939 9645 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.11.252" port: 41809 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:31:21.634635 9647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.634428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:41809 (local address 127.0.0.1:48678)
0808 22:31:21.634522 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:21.634573 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.252:41809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260808 22:31:21.635416 9647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.635154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:41809 (local address 127.0.0.1:48694)
0808 22:31:21.635310 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:21.635359 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.252:41809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
W20260808 22:31:21.635584 9645 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.11.252:41809: Network error: Client connection negotiation failed: client connection to 127.5.11.252:41809: connect: Connection refused (error 111)
Aug 08 22:31:21 dist-test-slave-7f01 krb5kdc[9495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228281, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.253@KRBTEST.COM for kudu/127.5.11.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.252.keytab.
Entry for principal kudu/127.5.11.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.252.keytab.
I20260808 22:31:21.641372 9781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.633549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40938 (local address 127.5.11.253:39887)
0808 22:31:21.633671 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:21.633675 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:21.633686 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:31:21.633712 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:21.633714 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:21.633754 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:21.633836 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:21.634241 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.634866 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.635632 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.636106 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.636211 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:21.638097 (+ 1886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:21.638100 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:21.638102 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:21.638122 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:21.639620 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:21.640118 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:21.640122 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:21.640124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:21.640208 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:21.640498 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:21.640502 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:21.640504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:21.640674 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:21.640785 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:21.641084 (+ 299us) server_negotiation.cc:299] Negotiation successful
0808 22:31:21.641216 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":29,"server-negotiator.queue_time_us":49}
I20260808 22:31:21.641372 9778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.633408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.253:39887 (local address 127.0.0.1:40938)
0808 22:31:21.633506 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:21.633517 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:21.633598 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:21.633846 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:21.633849 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:21.634005 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:21.634136 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:21.634142 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.634954 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.634961 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:21.635470 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:21.635479 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.636225 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.636228 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:21.636304 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:21.636722 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:21.636733 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:21.637972 (+ 1239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:21.639770 (+ 1798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:21.639774 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:21.639775 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:21.640013 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:21.640321 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:21.640324 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:21.640326 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:21.640406 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:21.640795 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:21.640800 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:21.640907 (+ 107us) client_negotiation.cc:769] Sending connection context
0808 22:31:21.641013 (+ 106us) client_negotiation.cc:240] Negotiation successful
0808 22:31:21.641156 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260808 22:31:21.642120 9775 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.11.252" port: 41809 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:31:21.642513 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.252:41809
--webserver_interface=127.5.11.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.252.keytab
--principal=kudu/127.5.11.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:45963
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:34561,127.5.11.253:39887,127.5.11.252:41809
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:31:21.642879 9778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.642541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:41809 (local address 127.0.0.1:48700)
0808 22:31:21.642685 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:21.642797 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.252:41809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 22:31:21.643097 9686 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.11.252:41809: connect: Connection refused (error 111)
I20260808 22:31:21.644032 9778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.643725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:41809 (local address 127.0.0.1:48704)
0808 22:31:21.643891 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:21.643953 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.252:41809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260808 22:31:21.644203 9775 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.11.252:41809: Network error: Client connection negotiation failed: client connection to 127.5.11.252:41809: connect: Connection refused (error 111)
I20260808 22:31:21.666775 9645 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.11.252" port: 41809 } attempt: 1
I20260808 22:31:21.667659 9647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.667306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:41809 (local address 127.0.0.1:48720)
0808 22:31:21.667471 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:21.667567 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.252:41809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260808 22:31:21.668473 9647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.668234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:41809 (local address 127.0.0.1:48734)
0808 22:31:21.668349 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:21.668404 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.252:41809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 22:31:21.668613 9645 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.11.252:41809: Network error: Client connection negotiation failed: client connection to 127.5.11.252:41809: connect: Connection refused (error 111)
I20260808 22:31:21.693486 9775 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.11.252" port: 41809 } attempt: 1
I20260808 22:31:21.694411 9778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.694038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:41809 (local address 127.0.0.1:48740)
0808 22:31:21.694222 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:21.694313 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.252:41809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260808 22:31:21.695186 9778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.694949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:41809 (local address 127.0.0.1:48756)
0808 22:31:21.695066 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:21.695120 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.252:41809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260808 22:31:21.695354 9775 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.11.252:41809: Network error: Client connection negotiation failed: client connection to 127.5.11.252:41809: connect: Connection refused (error 111)
I20260808 22:31:21.743875 9645 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.11.252" port: 41809 } attempt: 2
I20260808 22:31:21.744802 9647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.744441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:41809 (local address 127.0.0.1:48762)
0808 22:31:21.744623 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:21.744702 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.252:41809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 22:31:21.745672 9647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.745427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:41809 (local address 127.0.0.1:48776)
0808 22:31:21.745533 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:21.745580 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.252:41809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 22:31:21.745874 9645 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.11.252:41809: Network error: Client connection negotiation failed: client connection to 127.5.11.252:41809: connect: Connection refused (error 111)
W20260808 22:31:21.759819 9787 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 22:31:21.760072 9787 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:21.760136 9787 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:21.760190 9787 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260808 22:31:21.763581 9775 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.11.252" port: 41809 } attempt: 2
W20260808 22:31:21.764343 9787 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
I20260808 22:31:21.764374 9778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.764065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:41809 (local address 127.0.0.1:48788)
0808 22:31:21.764214 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:21.764277 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.252:41809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260808 22:31:21.764403 9787 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:21.764436 9787 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:21.764459 9787 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:31:21.764489 9787 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:31:21.765079 9778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.764859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:41809 (local address 127.0.0.1:48804)
0808 22:31:21.764970 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:21.765022 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.252:41809: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 22:31:21.765231 9775 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.11.252:41809: Network error: Client connection negotiation failed: client connection to 127.5.11.252:41809: connect: Connection refused (error 111)
I20260808 22:31:21.769814 9787 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:45963
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:34561,127.5.11.253:39887,127.5.11.252:41809
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.252.keytab
--principal=kudu/127.5.11.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.252:41809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.11.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9787
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:21.770802 9787 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:21.771723 9787 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:21.780061 9798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:21.780069 9795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:21.780180 9787 server_base.cc:1034] running on GCE node
W20260808 22:31:21.780069 9803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:21.780869 9787 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:21.781919 9787 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:21.783111 9787 hybrid_clock.cc:648] HybridClock initialized: now 1786228281783094 us; error 26 us; skew 500 ppm
Aug 08 22:31:21 dist-test-slave-7f01 krb5kdc[9495](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228281, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:21.785866 9787 init.cc:377] Logged in from keytab as kudu/127.5.11.252@KRBTEST.COM (short username kudu)
I20260808 22:31:21.786688 9787 webserver.cc:460] Webserver started at http://127.5.11.252:34989/ using document root <none> and password file <none>
I20260808 22:31:21.787302 9787 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:21.787395 9787 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:21.787623 9787 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:21.789676 9787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-2/data/instance:
uuid: "fedf3d93132948f08a9468e9446e8d8d"
format_stamp: "Formatted at 2026-08-08 22:31:21 on dist-test-slave-7f01"
server_key: "b741d2b1de67908983ec9e23fbe0ea1b"
server_key_iv: "e6ee5340a82135136621b1d192b9ccc8"
server_key_version: "encryptionkey@0"
I20260808 22:31:21.790246 9787 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-2/wal/instance:
uuid: "fedf3d93132948f08a9468e9446e8d8d"
format_stamp: "Formatted at 2026-08-08 22:31:21 on dist-test-slave-7f01"
server_key: "b741d2b1de67908983ec9e23fbe0ea1b"
server_key_iv: "e6ee5340a82135136621b1d192b9ccc8"
server_key_version: "encryptionkey@0"
I20260808 22:31:21.793998 9787 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260808 22:31:21.796401 9814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:21.797600 9787 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260808 22:31:21.797740 9787 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-2/wal
uuid: "fedf3d93132948f08a9468e9446e8d8d"
format_stamp: "Formatted at 2026-08-08 22:31:21 on dist-test-slave-7f01"
server_key: "b741d2b1de67908983ec9e23fbe0ea1b"
server_key_iv: "e6ee5340a82135136621b1d192b9ccc8"
server_key_version: "encryptionkey@0"
I20260808 22:31:21.797860 9787 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:21.807564 9787 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:21.810392 9787 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:21.810591 9787 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:21.817713 9787 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.252:41809
I20260808 22:31:21.818041 9901 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.252:41809 every 8 connection(s)
I20260808 22:31:21.818674 9787 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-2/data/info.pb
I20260808 22:31:21.819954 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 9787
I20260808 22:31:21.820096 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786227991988852-5167-0/minicluster-data/master-2/wal/instance
I20260808 22:31:21.820385 5167 external_mini_cluster.cc:1428] Setting key 9d6bf89bf44dbaa3a9c6b409d1cac031
I20260808 22:31:21.821923 9902 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:21.826246 9902 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.11.254" port: 34561 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 22:31:21 dist-test-slave-7f01 krb5kdc[9495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.252@KRBTEST.COM
Aug 08 22:31:21 dist-test-slave-7f01 krb5kdc[9495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228281, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.252@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:31:21.835875 9907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.821719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48820 (local address 127.5.11.252:41809)
0808 22:31:21.822124 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:21.822139 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:21.822178 (+ 39us) server_negotiation.cc:407] Connection header received
0808 22:31:21.822817 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:21.822833 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:21.823148 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:21.823421 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:21.824428 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.825426 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.826587 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.827131 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.827249 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:21.830580 (+ 3331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:21.830586 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:21.830594 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:21.830620 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:21.832543 (+ 1923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:21.833062 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:21.833067 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:21.833071 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:21.833177 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:21.833529 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:21.833540 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:21.833542 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:21.833960 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:21.834166 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:21.834477 (+ 311us) server_negotiation.cc:299] Negotiation successful
0808 22:31:21.834777 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":137,"threads_started":1}
I20260808 22:31:21.837450 9908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.828130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:34561 (local address 127.0.0.1:48268)
0808 22:31:21.828497 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:21.828561 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:21.828688 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:21.829045 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:21.829053 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:21.829529 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:21.829681 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:21.829687 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.830498 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.830502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:21.831277 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:21.831287 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.831773 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.831777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:21.831894 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:21.832365 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:21.832376 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:21.833780 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:21.835686 (+ 1906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:21.835695 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:21.835699 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:21.836048 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:21.836412 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:21.836416 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:21.836418 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:21.836513 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:21.836989 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:21.836993 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:21.837085 (+ 92us) client_negotiation.cc:769] Sending connection context
0808 22:31:21.837198 (+ 113us) client_negotiation.cc:240] Negotiation successful
0808 22:31:21.837323 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":95,"threads_started":1}
I20260808 22:31:21.837805 9648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.828217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48268 (local address 127.5.11.254:34561)
0808 22:31:21.828392 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:21.828397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:21.828648 (+ 251us) server_negotiation.cc:407] Connection header received
0808 22:31:21.828819 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:21.828821 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:21.828870 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:21.828965 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:21.829813 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.830369 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.831405 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.831627 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.831743 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:21.833921 (+ 2178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:21.833925 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:21.833928 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:21.833953 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:21.835574 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:21.836204 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:21.836208 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:21.836210 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:21.836306 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:21.836609 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:21.836613 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:21.836615 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:21.836867 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:21.836983 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:21.837284 (+ 301us) server_negotiation.cc:299] Negotiation successful
0808 22:31:21.837407 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260808 22:31:21.838824 9902 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.11.253" port: 39887 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 22:31:21 dist-test-slave-7f01 krb5kdc[9495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228281, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.252@KRBTEST.COM for kudu/127.5.11.253@KRBTEST.COM
Aug 08 22:31:21 dist-test-slave-7f01 krb5kdc[9495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:31:21.845046 9645 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.11.252" port: 41809 } attempt: 3
I20260808 22:31:21.847576 9781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.839416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40954 (local address 127.5.11.253:39887)
0808 22:31:21.839558 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:21.839562 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:21.839577 (+ 15us) server_negotiation.cc:407] Connection header received
0808 22:31:21.839622 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:21.839624 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:21.839661 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:21.839722 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:21.840189 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.840740 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.841492 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.841734 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.841841 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:21.843723 (+ 1882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:21.843726 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:21.843728 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:21.843790 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:21.845533 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:21.846056 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:21.846061 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:21.846064 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:21.846156 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:21.846506 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:21.846511 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:21.846516 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:21.846806 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:21.846902 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:21.847277 (+ 375us) server_negotiation.cc:299] Negotiation successful
0808 22:31:21.847409 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260808 22:31:21.848130 9908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.839265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.253:39887 (local address 127.0.0.1:40954)
0808 22:31:21.839373 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:21.839386 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:21.839462 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:21.839784 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:21.839787 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:21.839953 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:21.840056 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:21.840062 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.840855 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.840861 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:21.841364 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:21.841372 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.841853 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.841856 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:21.841962 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:21.842384 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:21.842394 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:21.843581 (+ 1187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:21.845626 (+ 2045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:21.845632 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:21.845637 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:21.845910 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:21.846286 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:21.846292 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:21.846295 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:21.846390 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:21.846949 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:21.846952 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:21.847036 (+ 84us) client_negotiation.cc:769] Sending connection context
0808 22:31:21.847859 (+ 823us) client_negotiation.cc:240] Negotiation successful
0808 22:31:21.847987 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260808 22:31:21.849112 9902 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.11.252" port: 41809 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:31:21.849207 9648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.838385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48272 (local address 127.5.11.254:34561)
0808 22:31:21.838497 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:21.838502 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:21.838561 (+ 59us) server_negotiation.cc:407] Connection header received
0808 22:31:21.838695 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:21.838698 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:21.838739 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:21.838827 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:21.839530 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.840242 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.841006 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.841231 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.841342 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:21.844464 (+ 3122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:21.844468 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:21.844470 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:21.844496 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:21.846615 (+ 2119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:21.847125 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:21.847129 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:21.847130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:21.847221 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:21.847518 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:21.847521 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:21.847523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:21.847744 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:21.848229 (+ 485us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:21.848896 (+ 667us) server_negotiation.cc:299] Negotiation successful
0808 22:31:21.849027 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
Aug 08 22:31:21 dist-test-slave-7f01 krb5kdc[9495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228281, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for kudu/127.5.11.252@KRBTEST.COM
Aug 08 22:31:21 dist-test-slave-7f01 krb5kdc[9495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228281, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.252@KRBTEST.COM for kudu/127.5.11.252@KRBTEST.COM
I20260808 22:31:21.854854 9907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.845715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48830 (local address 127.5.11.252:41809)
0808 22:31:21.845806 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:21.845810 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:21.845824 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:31:21.845857 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:21.845859 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:21.845888 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:21.845978 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:21.846556 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.847256 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.848461 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.848728 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.848832 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:21.851378 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:21.851381 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:21.851384 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:21.851406 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:21.853073 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:21.853543 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:21.853549 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:21.853554 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:21.853632 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:21.853980 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:21.853984 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:21.853986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:21.854200 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:21.854317 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:21.854607 (+ 290us) server_negotiation.cc:299] Negotiation successful
0808 22:31:21.854716 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260808 22:31:21.854875 9647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.845477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:41809 (local address 127.0.0.1:48830)
0808 22:31:21.845600 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:21.845615 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:21.845691 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:21.845999 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:21.846002 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:21.846327 (+ 325us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:21.846404 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:21.846410 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.847373 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.847379 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:21.848198 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:21.848213 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.848833 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.848837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:21.848918 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:21.849387 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:21.849405 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:21.851267 (+ 1862us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:21.853180 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:21.853183 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:21.853185 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:21.853462 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:21.853761 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:21.853764 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:21.853766 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:21.853859 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:21.854309 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:21.854312 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:21.854413 (+ 101us) client_negotiation.cc:769] Sending connection context
0808 22:31:21.854537 (+ 124us) client_negotiation.cc:240] Negotiation successful
0808 22:31:21.854684 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
Aug 08 22:31:21 dist-test-slave-7f01 krb5kdc[9495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228281, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.253@KRBTEST.COM
I20260808 22:31:21.856413 9775 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.11.252" port: 41809 } attempt: 3
I20260808 22:31:21.858668 9645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa6c968612454a12871737d4c2b5ffce: Bootstrap starting.
I20260808 22:31:21.860051 9908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.849567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:41809 (local address 127.0.0.1:48844)
0808 22:31:21.849689 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:21.849706 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:21.849784 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:21.850659 (+ 875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:21.850662 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:21.850782 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:21.850847 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:21.850852 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.851880 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.851883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:21.852707 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:21.852720 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.853324 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.853328 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:21.853435 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:21.853837 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:21.853850 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:21.855532 (+ 1682us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:21.858638 (+ 3106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:21.858641 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:21.858643 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:21.858869 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:21.859154 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:21.859157 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:21.859159 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:21.859234 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:21.859604 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:21.859607 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:21.859669 (+ 62us) client_negotiation.cc:769] Sending connection context
0808 22:31:21.859756 (+ 87us) client_negotiation.cc:240] Negotiation successful
0808 22:31:21.859871 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":20}
I20260808 22:31:21.860576 9909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.850093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48844 (local address 127.5.11.252:41809)
0808 22:31:21.850436 (+ 343us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:21.850445 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:21.850461 (+ 16us) server_negotiation.cc:407] Connection header received
0808 22:31:21.850497 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:21.850500 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:21.850549 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:21.850640 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:21.850968 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.851722 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.852843 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.853209 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.853324 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:21.855881 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:21.855884 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:21.855886 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:21.855906 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:21.858530 (+ 2624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:21.858977 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:21.858981 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:21.858982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:21.859069 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:21.859324 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:21.859327 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:21.859328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:21.859513 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:21.860230 (+ 717us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:21.860299 (+ 69us) server_negotiation.cc:299] Negotiation successful
0808 22:31:21.860401 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":274,"thread_start_us":150,"threads_started":1}
I20260808 22:31:21.861117 9781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.850954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40966 (local address 127.5.11.253:39887)
0808 22:31:21.851074 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:21.851078 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:21.851090 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:31:21.851121 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:21.851123 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:21.851163 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:21.851241 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:21.851978 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.852617 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.853498 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.853873 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.854052 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:21.856890 (+ 2838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:21.856894 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:21.856897 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:21.856931 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:21.858916 (+ 1985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:21.859390 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:21.859393 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:21.859395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:21.859507 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:21.859835 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:21.859838 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:21.859840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:21.860052 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:21.860293 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:21.860878 (+ 585us) server_negotiation.cc:299] Negotiation successful
0808 22:31:21.860988 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 22:31:21.861891 9645 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa6c968612454a12871737d4c2b5ffce: Neither blocks nor log segments found. Creating new log.
Aug 08 22:31:21 dist-test-slave-7f01 krb5kdc[9495](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228281, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.253@KRBTEST.COM for kudu/127.5.11.252@KRBTEST.COM
I20260808 22:31:21.862857 9645 log.cc:824] T 00000000000000000000000000000000 P fa6c968612454a12871737d4c2b5ffce: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:21.864930 5167 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:31:21.865996 9902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fedf3d93132948f08a9468e9446e8d8d: Bootstrap starting.
I20260808 22:31:21.866735 9778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.856782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:41809 (local address 127.0.0.1:48846)
0808 22:31:21.856950 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:21.856964 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:21.857053 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:21.857301 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:21.857305 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:21.857539 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:21.857629 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:21.857636 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.859050 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.859054 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:21.859886 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:21.859898 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.861077 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.861081 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:21.861176 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:21.861642 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:21.861659 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:21.863052 (+ 1393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:21.865041 (+ 1989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:21.865044 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:21.865046 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:21.865284 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:21.865610 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:21.865613 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:21.865615 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:21.865705 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:21.866224 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:21.866227 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:21.866328 (+ 101us) client_negotiation.cc:769] Sending connection context
0808 22:31:21.866445 (+ 117us) client_negotiation.cc:240] Negotiation successful
0808 22:31:21.866581 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96}
I20260808 22:31:21.866747 9907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.856819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48846 (local address 127.5.11.252:41809)
0808 22:31:21.856916 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:21.856920 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:21.857029 (+ 109us) server_negotiation.cc:407] Connection header received
0808 22:31:21.857125 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:21.857129 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:21.857175 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:21.857439 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:21.857849 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.858914 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.860036 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.860568 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.862403 (+ 1835us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:21.863203 (+ 800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:21.863207 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:21.863209 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:21.863235 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:21.864911 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:21.865407 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:21.865410 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:21.865412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:21.865505 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:21.865818 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:21.865821 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:21.865823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:21.866017 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:21.866201 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:21.866522 (+ 321us) server_negotiation.cc:299] Negotiation successful
0808 22:31:21.866627 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260808 22:31:21.866195 9591 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "fa6c968612454a12871737d4c2b5ffce"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48272
I20260808 22:31:21.868647 9902 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fedf3d93132948f08a9468e9446e8d8d: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:21.869518 9902 log.cc:824] T 00000000000000000000000000000000 P fedf3d93132948f08a9468e9446e8d8d: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:21.869545 9775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d527a8616eb646d98270bced5643ae1f: Bootstrap starting.
I20260808 22:31:21.871393 9645 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa6c968612454a12871737d4c2b5ffce: No bootstrap required, opened a new log
I20260808 22:31:21.871840 9775 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d527a8616eb646d98270bced5643ae1f: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:21.872498 9775 log.cc:824] T 00000000000000000000000000000000 P d527a8616eb646d98270bced5643ae1f: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:21.872619 9902 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fedf3d93132948f08a9468e9446e8d8d: No bootstrap required, opened a new log
I20260808 22:31:21.874513 9775 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d527a8616eb646d98270bced5643ae1f: No bootstrap required, opened a new log
I20260808 22:31:21.874492 9645 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa6c968612454a12871737d4c2b5ffce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa6c968612454a12871737d4c2b5ffce" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 34561 } } peers { permanent_uuid: "d527a8616eb646d98270bced5643ae1f" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 39887 } } peers { permanent_uuid: "fedf3d93132948f08a9468e9446e8d8d" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 41809 } }
I20260808 22:31:21.874724 9645 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa6c968612454a12871737d4c2b5ffce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa6c968612454a12871737d4c2b5ffce, State: Initialized, Role: FOLLOWER
I20260808 22:31:21.875072 9645 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa6c968612454a12871737d4c2b5ffce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa6c968612454a12871737d4c2b5ffce" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 34561 } } peers { permanent_uuid: "d527a8616eb646d98270bced5643ae1f" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 39887 } } peers { permanent_uuid: "fedf3d93132948f08a9468e9446e8d8d" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 41809 } }
I20260808 22:31:21.876430 9645 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa6c968612454a12871737d4c2b5ffce [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:31:21.876603 9902 raft_consensus.cc:348] T 00000000000000000000000000000000 P fedf3d93132948f08a9468e9446e8d8d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa6c968612454a12871737d4c2b5ffce" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 34561 } } peers { permanent_uuid: "d527a8616eb646d98270bced5643ae1f" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 39887 } } peers { permanent_uuid: "fedf3d93132948f08a9468e9446e8d8d" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 41809 } }
I20260808 22:31:21.876824 9902 raft_consensus.cc:729] T 00000000000000000000000000000000 P fedf3d93132948f08a9468e9446e8d8d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fedf3d93132948f08a9468e9446e8d8d, State: Initialized, Role: FOLLOWER
I20260808 22:31:21.877113 9902 consensus_queue.cc:260] T 00000000000000000000000000000000 P fedf3d93132948f08a9468e9446e8d8d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa6c968612454a12871737d4c2b5ffce" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 34561 } } peers { permanent_uuid: "d527a8616eb646d98270bced5643ae1f" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 39887 } } peers { permanent_uuid: "fedf3d93132948f08a9468e9446e8d8d" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 41809 } }
I20260808 22:31:21.878489 9902 sys_catalog.cc:564] T 00000000000000000000000000000000 P fedf3d93132948f08a9468e9446e8d8d [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:31:21.878985 9775 raft_consensus.cc:348] T 00000000000000000000000000000000 P d527a8616eb646d98270bced5643ae1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa6c968612454a12871737d4c2b5ffce" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 34561 } } peers { permanent_uuid: "d527a8616eb646d98270bced5643ae1f" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 39887 } } peers { permanent_uuid: "fedf3d93132948f08a9468e9446e8d8d" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 41809 } }
I20260808 22:31:21.879209 9775 raft_consensus.cc:729] T 00000000000000000000000000000000 P d527a8616eb646d98270bced5643ae1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d527a8616eb646d98270bced5643ae1f, State: Initialized, Role: FOLLOWER
I20260808 22:31:21.879526 9775 consensus_queue.cc:260] T 00000000000000000000000000000000 P d527a8616eb646d98270bced5643ae1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa6c968612454a12871737d4c2b5ffce" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 34561 } } peers { permanent_uuid: "d527a8616eb646d98270bced5643ae1f" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 39887 } } peers { permanent_uuid: "fedf3d93132948f08a9468e9446e8d8d" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 41809 } }
I20260808 22:31:21.881003 9775 sys_catalog.cc:564] T 00000000000000000000000000000000 P d527a8616eb646d98270bced5643ae1f [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:31:21.881495 9918 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa6c968612454a12871737d4c2b5ffce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa6c968612454a12871737d4c2b5ffce" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 34561 } } peers { permanent_uuid: "d527a8616eb646d98270bced5643ae1f" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 39887 } } peers { permanent_uuid: "fedf3d93132948f08a9468e9446e8d8d" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 41809 } } }
I20260808 22:31:21.881620 9918 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa6c968612454a12871737d4c2b5ffce [sys.catalog]: This master's current role is: FOLLOWER
W20260808 22:31:21.886371 9989 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d527a8616eb646d98270bced5643ae1f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:31:21.886451 9989 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:31:21.883666 9955 sys_catalog.cc:455] T 00000000000000000000000000000000 P d527a8616eb646d98270bced5643ae1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa6c968612454a12871737d4c2b5ffce" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 34561 } } peers { permanent_uuid: "d527a8616eb646d98270bced5643ae1f" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 39887 } } peers { permanent_uuid: "fedf3d93132948f08a9468e9446e8d8d" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 41809 } } }
I20260808 22:31:21.886700 9955 sys_catalog.cc:458] T 00000000000000000000000000000000 P d527a8616eb646d98270bced5643ae1f [sys.catalog]: This master's current role is: FOLLOWER
W20260808 22:31:21.887219 9965 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fedf3d93132948f08a9468e9446e8d8d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:31:21.887295 9965 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:31:21.888453 9591 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa6c968612454a12871737d4c2b5ffce [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 22:31:21.888576 9591 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa6c968612454a12871737d4c2b5ffce [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:21.890014 9591 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa6c968612454a12871737d4c2b5ffce [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa6c968612454a12871737d4c2b5ffce" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 34561 } } peers { permanent_uuid: "d527a8616eb646d98270bced5643ae1f" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 39887 } } peers { permanent_uuid: "fedf3d93132948f08a9468e9446e8d8d" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 41809 } }
I20260808 22:31:21.890692 9787 master_runner.cc:263] No leader master found from master fedf3d93132948f08a9468e9446e8d8d
I20260808 22:31:21.890758 9787 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 22:31:21.890884 9591 leader_election.cc:288] T 00000000000000000000000000000000 P fa6c968612454a12871737d4c2b5ffce [CANDIDATE]: Term 1 election: Requested vote from peers d527a8616eb646d98270bced5643ae1f (127.5.11.253:39887), fedf3d93132948f08a9468e9446e8d8d (127.5.11.252:41809)
I20260808 22:31:21.891574 9860 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fa6c968612454a12871737d4c2b5ffce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fedf3d93132948f08a9468e9446e8d8d"
I20260808 22:31:21.891848 9860 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fedf3d93132948f08a9468e9446e8d8d [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:21.892010 9734 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "fa6c968612454a12871737d4c2b5ffce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d527a8616eb646d98270bced5643ae1f"
I20260808 22:31:21.892234 9734 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d527a8616eb646d98270bced5643ae1f [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:21.893208 9860 raft_consensus.cc:2454] T 00000000000000000000000000000000 P fedf3d93132948f08a9468e9446e8d8d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa6c968612454a12871737d4c2b5ffce in term 1.
I20260808 22:31:21.894146 9734 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d527a8616eb646d98270bced5643ae1f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa6c968612454a12871737d4c2b5ffce in term 1.
W20260808 22:31:21.894691 9944 catalog_manager.cc:1558] T 00000000000000000000000000000000 P fa6c968612454a12871737d4c2b5ffce: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260808 22:31:21.895128 9524 master_runner.cc:263] No leader master found from master fa6c968612454a12871737d4c2b5ffce
I20260808 22:31:21.894958 9553 leader_election.cc:302] T 00000000000000000000000000000000 P fa6c968612454a12871737d4c2b5ffce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: fa6c968612454a12871737d4c2b5ffce, fedf3d93132948f08a9468e9446e8d8d; no voters:
W20260808 22:31:21.895169 9944 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:31:21.895188 9524 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 22:31:21.895674 9924 sys_catalog.cc:455] T 00000000000000000000000000000000 P fedf3d93132948f08a9468e9446e8d8d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa6c968612454a12871737d4c2b5ffce" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 34561 } } peers { permanent_uuid: "d527a8616eb646d98270bced5643ae1f" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 39887 } } peers { permanent_uuid: "fedf3d93132948f08a9468e9446e8d8d" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 41809 } } }
I20260808 22:31:21.895836 9924 sys_catalog.cc:458] T 00000000000000000000000000000000 P fedf3d93132948f08a9468e9446e8d8d [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:31:21.896101 9654 master_runner.cc:232] Existing masters have differing terms: 0 vs 1
I20260808 22:31:21.896157 9918 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa6c968612454a12871737d4c2b5ffce [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:21.896217 9654 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 22:31:21.896386 9918 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa6c968612454a12871737d4c2b5ffce [term 1 LEADER]: Becoming Leader. State: Replica: fa6c968612454a12871737d4c2b5ffce, State: Running, Role: LEADER
I20260808 22:31:21.896708 9918 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa6c968612454a12871737d4c2b5ffce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa6c968612454a12871737d4c2b5ffce" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 34561 } } peers { permanent_uuid: "d527a8616eb646d98270bced5643ae1f" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 39887 } } peers { permanent_uuid: "fedf3d93132948f08a9468e9446e8d8d" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 41809 } }
I20260808 22:31:21.898782 10005 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa6c968612454a12871737d4c2b5ffce [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa6c968612454a12871737d4c2b5ffce. Latest consensus state: current_term: 1 leader_uuid: "fa6c968612454a12871737d4c2b5ffce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa6c968612454a12871737d4c2b5ffce" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 34561 } } peers { permanent_uuid: "d527a8616eb646d98270bced5643ae1f" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 39887 } } peers { permanent_uuid: "fedf3d93132948f08a9468e9446e8d8d" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 41809 } } }
I20260808 22:31:21.898934 10005 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa6c968612454a12871737d4c2b5ffce [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:21.908042 10007 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:31:21.909143 10007 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:31:21.910266 9781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.900179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40978 (local address 127.5.11.253:39887)
0808 22:31:21.900299 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:21.900303 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:21.900498 (+ 195us) server_negotiation.cc:407] Connection header received
0808 22:31:21.900708 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:21.900714 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:21.900793 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:21.900853 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:21.901940 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.902690 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.903895 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.904247 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.904323 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:21.906200 (+ 1877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:21.906205 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:21.906208 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:21.906237 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:21.908034 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:21.908636 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:21.908643 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:21.908646 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:21.908739 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:21.909093 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:21.909102 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:21.909105 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:21.909431 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:21.909563 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:21.909964 (+ 401us) server_negotiation.cc:299] Negotiation successful
0808 22:31:21.910084 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 22:31:21.911778 9907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.900624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48856 (local address 127.5.11.252:41809)
0808 22:31:21.900738 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:21.900742 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:21.902224 (+ 1482us) server_negotiation.cc:407] Connection header received
0808 22:31:21.902378 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:21.902381 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:21.902422 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:21.902469 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:21.903404 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.904148 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.905261 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.905634 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.905721 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:21.908020 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:21.908023 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:21.908026 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:21.908050 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:21.909835 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:21.910324 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:21.910330 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:21.910332 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:21.910428 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:21.910742 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:21.910746 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:21.910747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:21.910978 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:21.911080 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:21.911443 (+ 363us) server_negotiation.cc:299] Negotiation successful
0808 22:31:21.911558 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 22:31:21.919170 9648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:21.901005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48284 (local address 127.5.11.254:34561)
0808 22:31:21.901113 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:21.901117 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:21.910993 (+ 9876us) server_negotiation.cc:407] Connection header received
0808 22:31:21.911291 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:21.911295 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:21.911352 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:21.911409 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:21.912464 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.913137 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.913859 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:21.914096 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:21.914204 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:21.915495 (+ 1291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:21.915499 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:21.915502 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:21.915528 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:21.917314 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:21.917808 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:21.917811 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:21.917813 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:21.917903 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:21.918221 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:21.918225 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:21.918226 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:21.918445 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:21.918543 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:21.918909 (+ 366us) server_negotiation.cc:299] Negotiation successful
0808 22:31:21.919023 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260808 22:31:21.927263 9860 raft_consensus.cc:1261] T 00000000000000000000000000000000 P fedf3d93132948f08a9468e9446e8d8d [term 1 FOLLOWER]: Refusing update from remote peer fa6c968612454a12871737d4c2b5ffce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:31:21.927997 9918 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fa6c968612454a12871737d4c2b5ffce [LEADER]: Connected to new peer: Peer: permanent_uuid: "fedf3d93132948f08a9468e9446e8d8d" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 41809 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:31:21.933689 9924 sys_catalog.cc:455] T 00000000000000000000000000000000 P fedf3d93132948f08a9468e9446e8d8d [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa6c968612454a12871737d4c2b5ffce. Latest consensus state: current_term: 1 leader_uuid: "fa6c968612454a12871737d4c2b5ffce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa6c968612454a12871737d4c2b5ffce" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 34561 } } peers { permanent_uuid: "d527a8616eb646d98270bced5643ae1f" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 39887 } } peers { permanent_uuid: "fedf3d93132948f08a9468e9446e8d8d" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 41809 } } }
I20260808 22:31:21.933807 9924 sys_catalog.cc:458] T 00000000000000000000000000000000 P fedf3d93132948f08a9468e9446e8d8d [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:31:21.935040 9734 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d527a8616eb646d98270bced5643ae1f [term 1 FOLLOWER]: Refusing update from remote peer fa6c968612454a12871737d4c2b5ffce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:31:21.935618 9918 consensus_queue.cc:1035] T 00000000000000000000000000000000 P fa6c968612454a12871737d4c2b5ffce [LEADER]: Connected to new peer: Peer: permanent_uuid: "d527a8616eb646d98270bced5643ae1f" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 39887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:31:21.938575 9918 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa6c968612454a12871737d4c2b5ffce [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fa6c968612454a12871737d4c2b5ffce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa6c968612454a12871737d4c2b5ffce" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 34561 } } peers { permanent_uuid: "d527a8616eb646d98270bced5643ae1f" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 39887 } } peers { permanent_uuid: "fedf3d93132948f08a9468e9446e8d8d" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 41809 } } }
I20260808 22:31:21.938697 9918 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa6c968612454a12871737d4c2b5ffce [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:21.939172 9955 sys_catalog.cc:455] T 00000000000000000000000000000000 P d527a8616eb646d98270bced5643ae1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa6c968612454a12871737d4c2b5ffce. Latest consensus state: current_term: 1 leader_uuid: "fa6c968612454a12871737d4c2b5ffce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa6c968612454a12871737d4c2b5ffce" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 34561 } } peers { permanent_uuid: "d527a8616eb646d98270bced5643ae1f" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 39887 } } peers { permanent_uuid: "fedf3d93132948f08a9468e9446e8d8d" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 41809 } } }
I20260808 22:31:21.939296 9955 sys_catalog.cc:458] T 00000000000000000000000000000000 P d527a8616eb646d98270bced5643ae1f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:31:21.941859 9924 sys_catalog.cc:455] T 00000000000000000000000000000000 P fedf3d93132948f08a9468e9446e8d8d [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fa6c968612454a12871737d4c2b5ffce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa6c968612454a12871737d4c2b5ffce" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 34561 } } peers { permanent_uuid: "d527a8616eb646d98270bced5643ae1f" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 39887 } } peers { permanent_uuid: "fedf3d93132948f08a9468e9446e8d8d" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 41809 } } }
I20260808 22:31:21.941974 9924 sys_catalog.cc:458] T 00000000000000000000000000000000 P fedf3d93132948f08a9468e9446e8d8d [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:31:21.942281 10012 mvcc.cc:205] Tried to move back new op lower bound from 7316391042765209600 to 7316391042652172288. Current Snapshot: MvccSnapshot[applied={T|T < 7316391042765209600}]
I20260808 22:31:21.943224 10015 mvcc.cc:205] Tried to move back new op lower bound from 7316391042765209600 to 7316391042652172288. Current Snapshot: MvccSnapshot[applied={T|T < 7316391042765209600}]
I20260808 22:31:21.945112 10007 catalog_manager.cc:1347] Generated new cluster ID: d517192d6a534086b9fc1c86204f1a61
I20260808 22:31:21.945182 10007 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:31:21.956259 10005 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa6c968612454a12871737d4c2b5ffce [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fa6c968612454a12871737d4c2b5ffce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa6c968612454a12871737d4c2b5ffce" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 34561 } } peers { permanent_uuid: "d527a8616eb646d98270bced5643ae1f" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 39887 } } peers { permanent_uuid: "fedf3d93132948f08a9468e9446e8d8d" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 41809 } } }
I20260808 22:31:21.956393 10005 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa6c968612454a12871737d4c2b5ffce [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:21.957510 9955 sys_catalog.cc:455] T 00000000000000000000000000000000 P d527a8616eb646d98270bced5643ae1f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fa6c968612454a12871737d4c2b5ffce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fa6c968612454a12871737d4c2b5ffce" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 34561 } } peers { permanent_uuid: "d527a8616eb646d98270bced5643ae1f" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 39887 } } peers { permanent_uuid: "fedf3d93132948f08a9468e9446e8d8d" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 41809 } } }
I20260808 22:31:21.957633 9955 sys_catalog.cc:458] T 00000000000000000000000000000000 P d527a8616eb646d98270bced5643ae1f [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:31:21.957962 10018 mvcc.cc:205] Tried to move back new op lower bound from 7316391042765209600 to 7316391042652172288. Current Snapshot: MvccSnapshot[applied={T|T < 7316391042765209600}]
I20260808 22:31:21.968055 10007 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:31:21.969352 10007 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:31:21.984651 10007 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa6c968612454a12871737d4c2b5ffce: Generated new TSK 0
I20260808 22:31:21.987665 10007 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 22:31:22.028131 9648 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:31:22.024771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48294 (local address 127.5.11.254:34561)
0808 22:31:22.024905 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:22.024910 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:22.026980 (+ 2070us) server_negotiation.cc:407] Connection header received
0808 22:31:22.027195 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:22.027199 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:22.027255 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:22.027309 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:22.027986 (+ 677us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48294: BlockingRecv error: recv got EOF from 127.0.0.1:48294 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260808 22:31:22.029536 9781 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:31:22.025290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40980 (local address 127.5.11.253:39887)
0808 22:31:22.025406 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:22.025409 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:22.028634 (+ 3225us) server_negotiation.cc:407] Connection header received
0808 22:31:22.028813 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:22.028816 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:22.028866 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:22.028916 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:22.029403 (+ 487us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40980: BlockingRecv error: recv got EOF from 127.0.0.1:40980 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 22:31:22.030886 9907 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:31:22.026324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48860 (local address 127.5.11.252:41809)
0808 22:31:22.026437 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:22.026441 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:22.029993 (+ 3552us) server_negotiation.cc:407] Connection header received
0808 22:31:22.030170 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:22.030174 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:22.030221 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:22.030271 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:22.030765 (+ 494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48860: BlockingRecv error: recv got EOF from 127.0.0.1:48860 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 22:31:22.035141 9781 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:31:22.033810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40988 (local address 127.5.11.253:39887)
0808 22:31:22.033930 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:22.033934 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:22.034189 (+ 255us) server_negotiation.cc:407] Connection header received
0808 22:31:22.034418 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:22.034422 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:22.034495 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:22.034558 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:22.035047 (+ 489us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40988: BlockingRecv error: recv got EOF from 127.0.0.1:40988 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 22:31:22.040048 9781 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:31:22.038631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40994 (local address 127.5.11.253:39887)
0808 22:31:22.038720 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:22.038724 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:22.039209 (+ 485us) server_negotiation.cc:407] Connection header received
0808 22:31:22.039371 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:22.039374 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:22.039417 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:22.039463 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:22.039957 (+ 494us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40994: BlockingRecv error: recv got EOF from 127.0.0.1:40994 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260808 22:31:22.045778 9781 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:31:22.044242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41000 (local address 127.5.11.253:39887)
0808 22:31:22.044337 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:22.044341 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:22.044863 (+ 522us) server_negotiation.cc:407] Connection header received
0808 22:31:22.045044 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:22.045047 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:22.045093 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:22.045142 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:22.045687 (+ 545us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41000: BlockingRecv error: recv got EOF from 127.0.0.1:41000 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260808 22:31:22.053694 9781 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:31:22.052010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41014 (local address 127.5.11.253:39887)
0808 22:31:22.052137 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:22.052141 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:22.052733 (+ 592us) server_negotiation.cc:407] Connection header received
0808 22:31:22.052932 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:22.052935 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:22.052981 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:22.053030 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:22.053585 (+ 555us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41014: BlockingRecv error: recv got EOF from 127.0.0.1:41014 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
W20260808 22:31:22.066679 9781 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:31:22.064901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41024 (local address 127.5.11.253:39887)
0808 22:31:22.065020 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:22.065026 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:22.065660 (+ 634us) server_negotiation.cc:407] Connection header received
0808 22:31:22.065884 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:22.065888 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:22.065943 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:22.065995 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:22.066561 (+ 566us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41024: BlockingRecv error: recv got EOF from 127.0.0.1:41024 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 22:31:22.087072 9781 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:31:22.085385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41040 (local address 127.5.11.253:39887)
0808 22:31:22.085523 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:22.085527 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:22.086098 (+ 571us) server_negotiation.cc:407] Connection header received
0808 22:31:22.086291 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:22.086295 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:22.086345 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:22.086396 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:22.086955 (+ 559us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41040: BlockingRecv error: recv got EOF from 127.0.0.1:41040 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260808 22:31:22.122853 9781 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:31:22.121525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41056 (local address 127.5.11.253:39887)
0808 22:31:22.121666 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:22.121672 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:22.121874 (+ 202us) server_negotiation.cc:407] Connection header received
0808 22:31:22.122062 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:22.122070 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:22.122121 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:22.122216 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:22.122702 (+ 486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41056: BlockingRecv error: recv got EOF from 127.0.0.1:41056 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260808 22:31:22.190340 9781 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:31:22.189149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41066 (local address 127.5.11.253:39887)
0808 22:31:22.189313 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:22.189318 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:22.189481 (+ 163us) server_negotiation.cc:407] Connection header received
0808 22:31:22.189609 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:22.189612 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:22.189665 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:22.189741 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:22.190202 (+ 461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41066: BlockingRecv error: recv got EOF from 127.0.0.1:41066 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260808 22:31:22.321887 9781 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:31:22.320612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41074 (local address 127.5.11.253:39887)
0808 22:31:22.320765 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:22.320771 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:22.320921 (+ 150us) server_negotiation.cc:407] Connection header received
0808 22:31:22.321061 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:22.321064 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:22.321116 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:22.321191 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:22.321747 (+ 556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41074: BlockingRecv error: recv got EOF from 127.0.0.1:41074 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260808 22:31:22.581565 9781 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:31:22.580362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41088 (local address 127.5.11.253:39887)
0808 22:31:22.580544 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:22.580551 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:22.580569 (+ 18us) server_negotiation.cc:407] Connection header received
0808 22:31:22.580787 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:22.580790 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:22.580842 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:22.580936 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:22.581451 (+ 515us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41088: BlockingRecv error: recv got EOF from 127.0.0.1:41088 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260808 22:31:22.887827 9989 catalog_manager.cc:1259] Loaded cluster ID: d517192d6a534086b9fc1c86204f1a61
I20260808 22:31:22.887945 9989 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d527a8616eb646d98270bced5643ae1f: loading cluster ID for follower catalog manager: success
I20260808 22:31:22.888477 9965 catalog_manager.cc:1259] Loaded cluster ID: d517192d6a534086b9fc1c86204f1a61
I20260808 22:31:22.888537 9965 catalog_manager.cc:1552] T 00000000000000000000000000000000 P fedf3d93132948f08a9468e9446e8d8d: loading cluster ID for follower catalog manager: success
I20260808 22:31:22.889671 9989 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d527a8616eb646d98270bced5643ae1f: acquiring CA information for follower catalog manager: success
I20260808 22:31:22.890105 9965 catalog_manager.cc:1574] T 00000000000000000000000000000000 P fedf3d93132948f08a9468e9446e8d8d: acquiring CA information for follower catalog manager: success
I20260808 22:31:22.890893 9989 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d527a8616eb646d98270bced5643ae1f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 22:31:22.891235 9965 catalog_manager.cc:1602] T 00000000000000000000000000000000 P fedf3d93132948f08a9468e9446e8d8d: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 22:31:23.099720 10156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:23.095934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41100 (local address 127.5.11.253:39887)
0808 22:31:23.096219 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:23.096223 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:23.096286 (+ 63us) server_negotiation.cc:407] Connection header received
0808 22:31:23.096427 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:23.096430 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:23.096568 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:23.096674 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:31:23.096924 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:23.097444 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:23.098266 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:23.098487 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:23.098563 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:23.098836 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:31:23.099288 (+ 452us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:31:23.099381 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:23.099460 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 22:31:23.099508 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":92,"threads_started":1}
I20260808 22:31:23.106156 10166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:23.102760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48870 (local address 127.5.11.252:41809)
0808 22:31:23.102985 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:23.102989 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:23.103002 (+ 13us) server_negotiation.cc:407] Connection header received
0808 22:31:23.103130 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:23.103132 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:23.103243 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:23.103340 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:31:23.103534 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:23.104097 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:23.104860 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:23.105082 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:23.105159 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:23.105429 (+ 270us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:31:23.105842 (+ 413us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:31:23.105932 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:23.105996 (+ 64us) server_negotiation.cc:299] Negotiation successful
0808 22:31:23.106040 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":81,"threads_started":1}
I20260808 22:31:23.113220 10156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:23.109259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41116 (local address 127.5.11.253:39887)
0808 22:31:23.109379 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:23.109383 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:23.109810 (+ 427us) server_negotiation.cc:407] Connection header received
0808 22:31:23.109903 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:23.109907 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:23.109971 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:23.110046 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:31:23.110294 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:23.111028 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:23.111927 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:23.112261 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:23.112363 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:23.112732 (+ 369us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:31:23.112807 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:31:23.112875 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:23.113004 (+ 129us) server_negotiation.cc:299] Negotiation successful
0808 22:31:23.113058 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260808 22:31:23.114353 10166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:23.109674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48876 (local address 127.5.11.252:41809)
0808 22:31:23.109776 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:23.109780 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:23.110334 (+ 554us) server_negotiation.cc:407] Connection header received
0808 22:31:23.110492 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:23.110495 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:23.110545 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:23.110588 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:31:23.110921 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:23.111629 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:23.112919 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:23.113239 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:23.113332 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:23.113656 (+ 324us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:31:23.113723 (+ 67us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:31:23.113766 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:23.114185 (+ 419us) server_negotiation.cc:299] Negotiation successful
0808 22:31:23.114233 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260808 22:31:23.114367 10182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:23.109246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48304 (local address 127.5.11.254:34561)
0808 22:31:23.109633 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:23.109638 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:23.110044 (+ 406us) server_negotiation.cc:407] Connection header received
0808 22:31:23.110167 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:23.110170 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:23.110329 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:23.110439 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:31:23.110666 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:23.111448 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:23.112748 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:23.113114 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:23.113205 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:23.113607 (+ 402us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:31:23.113916 (+ 309us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:31:23.114005 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:23.114175 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 22:31:23.114235 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":76,"threads_started":1}
I20260808 22:31:23.117250 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 9524
I20260808 22:31:23.133416 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 9654
I20260808 22:31:23.145646 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 9787
2026-08-08T22:31:23Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3382 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 08 22:31:23 dist-test-slave-7f01 krb5kdc[10196](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 08 22:31:23 dist-test-slave-7f01 krb5kdc[10196](info): set up 2 sockets
Aug 08 22:31:23 dist-test-slave-7f01 krb5kdc[10196](info): commencing operation
krb5kdc: starting...
W20260808 22:31:24.407645 5167 mini_kdc.cc:121] Time spent starting KDC: real 1.233s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/krb5kdc/test-user.keytab.
Aug 08 22:31:24 dist-test-slave-7f01 krb5kdc[10196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-08T22:31:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-08T22:31:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.5.11.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
Entry for principal kudu/127.5.11.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab.
I20260808 22:31:24.534067 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:45687
--webserver_interface=127.5.11.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:44039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:45687,127.5.11.253:40863,127.5.11.252:36887
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260808 22:31:24.649530 10222 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 22:31:24.649823 10222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:24.649887 10222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:24.649962 10222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:24.653985 10222 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:31:24.654042 10222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:24.654062 10222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:24.654120 10222 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:31:24.654157 10222 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:31:24.659338 10222 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:44039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:45687,127.5.11.253:40863,127.5.11.252:36887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.254.keytab
--principal=kudu/127.5.11.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.254:45687
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.5.11.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:24.660455 10222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:24.661499 10222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:24.670142 10231 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:24.670239 10227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:24.670140 10234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:24.670480 10222 server_base.cc:1034] running on GCE node
I20260808 22:31:24.671108 10222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:24.672282 10222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:24.673496 10222 hybrid_clock.cc:648] HybridClock initialized: now 1786228284673469 us; error 46 us; skew 500 ppm
Aug 08 22:31:24 dist-test-slave-7f01 krb5kdc[10196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228284, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:24.676338 10222 init.cc:377] Logged in from keytab as kudu/127.5.11.254@KRBTEST.COM (short username kudu)
I20260808 22:31:24.677261 10222 webserver.cc:460] Webserver started at http://127.5.11.254:41595/ using document root <none> and password file <none>
I20260808 22:31:24.677834 10222 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:24.677897 10222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:24.678148 10222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:24.680255 10222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-0/data/instance:
uuid: "42f3c83e63004f3baabecd7b35170fa6"
format_stamp: "Formatted at 2026-08-08 22:31:24 on dist-test-slave-7f01"
server_key: "66398e6176a7c48a97402b87d0b665f6"
server_key_iv: "390c33dc6153107c31351b71d21d22cf"
server_key_version: "encryptionkey@0"
I20260808 22:31:24.680848 10222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-0/wal/instance:
uuid: "42f3c83e63004f3baabecd7b35170fa6"
format_stamp: "Formatted at 2026-08-08 22:31:24 on dist-test-slave-7f01"
server_key: "66398e6176a7c48a97402b87d0b665f6"
server_key_iv: "390c33dc6153107c31351b71d21d22cf"
server_key_version: "encryptionkey@0"
I20260808 22:31:24.684581 10222 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260808 22:31:24.687044 10244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:24.688318 10222 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260808 22:31:24.688467 10222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-0/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-0/wal
uuid: "42f3c83e63004f3baabecd7b35170fa6"
format_stamp: "Formatted at 2026-08-08 22:31:24 on dist-test-slave-7f01"
server_key: "66398e6176a7c48a97402b87d0b665f6"
server_key_iv: "390c33dc6153107c31351b71d21d22cf"
server_key_version: "encryptionkey@0"
I20260808 22:31:24.688565 10222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:24.699489 10222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:24.702313 10222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:24.702539 10222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:24.709651 10222 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.254:45687
I20260808 22:31:24.709949 10341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.254:45687 every 8 connection(s)
I20260808 22:31:24.710702 10222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-0/data/info.pb
I20260808 22:31:24.713574 10342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:24.717259 10342 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.11.254" port: 45687 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:31:24.720748 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 10222
I20260808 22:31:24.720906 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-0/wal/instance
I20260808 22:31:24.721194 5167 external_mini_cluster.cc:1428] Setting key 4c13a44b5c8deea0bd6a01adfa9c4fdc
Aug 08 22:31:24 dist-test-slave-7f01 krb5kdc[10196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228284, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.11.253@KRBTEST.COM; defaulting to no policy
I20260808 22:31:24.732697 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:24.720187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49164 (local address 127.5.11.254:45687)
0808 22:31:24.720494 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:24.720504 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:24.720550 (+ 46us) server_negotiation.cc:407] Connection header received
0808 22:31:24.721197 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:24.721201 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:24.721426 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:24.721619 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:24.722557 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:24.723393 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:24.724456 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:24.724837 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:24.725065 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:24.727947 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:24.727953 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:24.727962 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:24.727986 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:24.729613 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:24.730210 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:24.730215 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:24.730216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:24.730332 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:24.730699 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:24.730702 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:24.730703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:24.731136 (+ 433us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:24.731241 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:24.731641 (+ 400us) server_negotiation.cc:299] Negotiation successful
0808 22:31:24.731918 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":33,"server-negotiator.queue_time_us":190,"thread_start_us":78,"threads_started":1}
I20260808 22:31:24.732697 10345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:24.719370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:45687 (local address 127.0.0.1:49164)
0808 22:31:24.719888 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:24.719930 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:24.720806 (+ 876us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:24.721619 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:24.721624 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:24.722084 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:24.722399 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:24.722415 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:24.723537 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:24.723541 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:24.724332 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:24.724343 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:24.724992 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:24.724997 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:24.725166 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:24.725806 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:24.725823 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:24.727791 (+ 1968us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:24.729734 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:24.729744 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:24.729754 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:24.730089 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:24.730439 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:24.730443 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:24.730445 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:24.730598 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:24.731253 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:24.731258 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:24.731397 (+ 139us) client_negotiation.cc:769] Sending connection context
0808 22:31:24.731538 (+ 141us) client_negotiation.cc:240] Negotiation successful
0808 22:31:24.731997 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"mutex_wait_us":176,"thread_start_us":122,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.253@KRBTEST.COM" created.
I20260808 22:31:24.735459 10342 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.11.253" port: 40863 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:31:24.736311 10345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:24.736068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.253:40863 (local address 127.0.0.1:59792)
0808 22:31:24.736185 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:24.736254 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.253:40863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260808 22:31:24.736533 10258 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.11.253:40863: connect: Connection refused (error 111)
I20260808 22:31:24.737325 10345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:24.737125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.253:40863 (local address 127.0.0.1:59804)
0808 22:31:24.737230 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:24.737278 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.253:40863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 22:31:24.737565 10342 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.11.253:40863: Network error: Client connection negotiation failed: client connection to 127.5.11.253:40863: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.253.keytab.
Entry for principal kudu/127.5.11.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.253.keytab.
I20260808 22:31:24.750142 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.253:40863
--webserver_interface=127.5.11.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.253.keytab
--principal=kudu/127.5.11.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:44039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:45687,127.5.11.253:40863,127.5.11.252:36887
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:31:24.786944 10342 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.11.253" port: 40863 } attempt: 1
I20260808 22:31:24.787964 10345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:24.787607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.253:40863 (local address 127.0.0.1:59810)
0808 22:31:24.787821 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:24.787885 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.253:40863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260808 22:31:24.788707 10345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:24.788490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.253:40863 (local address 127.0.0.1:59814)
0808 22:31:24.788602 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:24.788653 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.253:40863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 22:31:24.788831 10342 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.11.253:40863: Network error: Client connection negotiation failed: client connection to 127.5.11.253:40863: connect: Connection refused (error 111)
I20260808 22:31:24.857028 10342 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.11.253" port: 40863 } attempt: 2
I20260808 22:31:24.857926 10345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:24.857616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.253:40863 (local address 127.0.0.1:59816)
0808 22:31:24.857786 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:24.857847 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.253:40863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260808 22:31:24.858659 10345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:24.858438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.253:40863 (local address 127.0.0.1:59824)
0808 22:31:24.858553 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:24.858605 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.253:40863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260808 22:31:24.858788 10342 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.11.253:40863: Network error: Client connection negotiation failed: client connection to 127.5.11.253:40863: connect: Connection refused (error 111)
W20260808 22:31:24.863893 10357 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 22:31:24.864192 10357 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:24.864253 10357 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:24.864331 10357 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:24.868323 10357 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:31:24.868400 10357 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:24.868448 10357 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:24.868505 10357 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:31:24.868542 10357 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:31:24.873800 10357 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:44039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:45687,127.5.11.253:40863,127.5.11.252:36887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.253.keytab
--principal=kudu/127.5.11.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.253:40863
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.5.11.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10357
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:24.874886 10357 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:24.875972 10357 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:24.884405 10366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:24.884351 10364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:24.884357 10369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:24.884516 10357 server_base.cc:1034] running on GCE node
I20260808 22:31:24.885232 10357 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:24.886333 10357 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:24.887526 10357 hybrid_clock.cc:648] HybridClock initialized: now 1786228284887493 us; error 43 us; skew 500 ppm
Aug 08 22:31:24 dist-test-slave-7f01 krb5kdc[10196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228284, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:24.890482 10357 init.cc:377] Logged in from keytab as kudu/127.5.11.253@KRBTEST.COM (short username kudu)
I20260808 22:31:24.891381 10357 webserver.cc:460] Webserver started at http://127.5.11.253:43155/ using document root <none> and password file <none>
I20260808 22:31:24.891989 10357 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:24.892055 10357 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:24.892223 10357 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:24.894322 10357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-1/data/instance:
uuid: "394cdb68baf744298ee7068eb138e07c"
format_stamp: "Formatted at 2026-08-08 22:31:24 on dist-test-slave-7f01"
server_key: "8e5522ad24cfc1d934ecb1e321616a1c"
server_key_iv: "45a2b3ba7603dfb0264035f3a0a93f20"
server_key_version: "encryptionkey@0"
I20260808 22:31:24.894907 10357 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-1/wal/instance:
uuid: "394cdb68baf744298ee7068eb138e07c"
format_stamp: "Formatted at 2026-08-08 22:31:24 on dist-test-slave-7f01"
server_key: "8e5522ad24cfc1d934ecb1e321616a1c"
server_key_iv: "45a2b3ba7603dfb0264035f3a0a93f20"
server_key_version: "encryptionkey@0"
I20260808 22:31:24.898588 10357 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 22:31:24.901031 10379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:24.902215 10357 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:31:24.902344 10357 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-1/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-1/wal
uuid: "394cdb68baf744298ee7068eb138e07c"
format_stamp: "Formatted at 2026-08-08 22:31:24 on dist-test-slave-7f01"
server_key: "8e5522ad24cfc1d934ecb1e321616a1c"
server_key_iv: "45a2b3ba7603dfb0264035f3a0a93f20"
server_key_version: "encryptionkey@0"
I20260808 22:31:24.902448 10357 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:24.916013 10357 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:24.918802 10357 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:24.919023 10357 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:24.926620 10357 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.253:40863
I20260808 22:31:24.926919 10463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.253:40863 every 8 connection(s)
I20260808 22:31:24.927650 10357 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-1/data/info.pb
I20260808 22:31:24.930609 10464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:24.934424 10464 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.11.254" port: 45687 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:31:24.936930 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 10357
I20260808 22:31:24.937108 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-1/wal/instance
I20260808 22:31:24.937462 5167 external_mini_cluster.cc:1428] Setting key a47f08870ee5ebf31ec69bc90b4b4036
Aug 08 22:31:24 dist-test-slave-7f01 krb5kdc[10196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228284, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.253@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.5.11.252@KRBTEST.COM; defaulting to no policy
I20260808 22:31:24.948941 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:24.936315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49180 (local address 127.5.11.254:45687)
0808 22:31:24.936500 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:24.936505 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:24.937349 (+ 844us) server_negotiation.cc:407] Connection header received
0808 22:31:24.938285 (+ 936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:24.938292 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:24.938363 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:24.938473 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:24.939627 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:24.940225 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:24.941881 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:24.942120 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:24.942239 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:24.945239 (+ 3000us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:24.945243 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:24.945245 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:24.945269 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:24.946800 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:24.947447 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:24.947450 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:24.947451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:24.947529 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:24.947940 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:24.947946 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:24.947950 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:24.948164 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:24.948246 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:24.948722 (+ 476us) server_negotiation.cc:299] Negotiation successful
0808 22:31:24.948821 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.5.11.252@KRBTEST.COM" created.
I20260808 22:31:24.949859 10466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:24.936686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:45687 (local address 127.0.0.1:49180)
0808 22:31:24.937171 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:24.937224 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:24.938073 (+ 849us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:24.938626 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:24.938632 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:24.938985 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:24.939464 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:24.939477 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:24.940372 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:24.940375 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:24.941743 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:24.941755 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:24.942288 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:24.942293 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:24.942458 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:24.943081 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:24.943098 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:24.944984 (+ 1886us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:24.946925 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:24.946933 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:24.946950 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:24.947331 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:24.947618 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:24.947620 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:24.947621 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:24.947823 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:24.948260 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:24.948265 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:24.948503 (+ 238us) client_negotiation.cc:769] Sending connection context
0808 22:31:24.948658 (+ 155us) client_negotiation.cc:240] Negotiation successful
0808 22:31:24.948895 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":146,"threads_started":1}
I20260808 22:31:24.950042 10342 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.11.253" port: 40863 } attempt: 3
I20260808 22:31:24.952037 10464 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.11.253" port: 40863 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 22:31:24 dist-test-slave-7f01 krb5kdc[10196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228284, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for kudu/127.5.11.253@KRBTEST.COM
Aug 08 22:31:24 dist-test-slave-7f01 krb5kdc[10196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228284, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.253@KRBTEST.COM for kudu/127.5.11.253@KRBTEST.COM
I20260808 22:31:24.961812 10345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:24.950483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.253:40863 (local address 127.0.0.1:59832)
0808 22:31:24.950697 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:24.950714 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:24.950793 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:24.951931 (+ 1138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:24.951935 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:24.952140 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:24.952264 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:24.952273 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:24.953639 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:24.953642 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:24.954416 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:24.954425 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:24.955045 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:24.955049 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:24.955156 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:24.955643 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:24.955660 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:24.957315 (+ 1655us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:24.960071 (+ 2756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:24.960075 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:24.960077 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:24.960361 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:24.960714 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:24.960721 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:24.960723 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:24.960816 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:24.961284 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:24.961287 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:24.961384 (+ 97us) client_negotiation.cc:769] Sending connection context
0808 22:31:24.961510 (+ 126us) client_negotiation.cc:240] Negotiation successful
0808 22:31:24.961643 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116}
I20260808 22:31:24.961896 10468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:24.951128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59832 (local address 127.5.11.253:40863)
0808 22:31:24.951481 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:24.951493 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:24.951513 (+ 20us) server_negotiation.cc:407] Connection header received
0808 22:31:24.951547 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:24.951552 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:24.951721 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:24.952430 (+ 709us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:24.952558 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:24.953518 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:24.954533 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:24.954922 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:24.955048 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:24.957451 (+ 2403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:24.957461 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:24.957483 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:24.957509 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:24.959671 (+ 2162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:24.960491 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:24.960501 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:24.960504 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:24.960593 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:24.960912 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:24.960917 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:24.960919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:24.961171 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:24.961317 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:24.961600 (+ 283us) server_negotiation.cc:299] Negotiation successful
0808 22:31:24.961735 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":265,"thread_start_us":88,"threads_started":1}
I20260808 22:31:24.962450 10466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:24.952546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.253:40863 (local address 127.0.0.1:59848)
0808 22:31:24.952707 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:24.952723 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:24.952797 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:24.953051 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:24.953054 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:24.953225 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:24.953298 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:24.953303 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:24.954311 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:24.954317 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:24.955357 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:24.955373 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:24.956015 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:24.956019 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:24.956102 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:24.956538 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:24.956552 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:24.957986 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:24.960658 (+ 2672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:24.960661 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:24.960662 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:24.960843 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:24.961209 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:24.961212 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:24.961214 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:24.961296 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:24.961763 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:24.961766 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:24.961851 (+ 85us) client_negotiation.cc:769] Sending connection context
0808 22:31:24.962104 (+ 253us) client_negotiation.cc:240] Negotiation successful
0808 22:31:24.962264 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":55}
I20260808 22:31:24.962529 10471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:24.952567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59848 (local address 127.5.11.253:40863)
0808 22:31:24.952856 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:24.952865 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:24.952877 (+ 12us) server_negotiation.cc:407] Connection header received
0808 22:31:24.952912 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:24.952914 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:24.952944 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:24.953041 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:24.953405 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:24.954187 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:24.955530 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:24.955881 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:24.956015 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:24.958159 (+ 2144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:24.958165 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:24.958168 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:24.958195 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:24.960552 (+ 2357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:24.961024 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:24.961027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:24.961028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:24.961104 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:24.961395 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:24.961400 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:24.961402 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:24.961664 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:24.961774 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:24.962152 (+ 378us) server_negotiation.cc:299] Negotiation successful
0808 22:31:24.962334 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":222,"spinlock_wait_cycles":6016,"thread_start_us":95,"threads_started":1}
I20260808 22:31:24.963797 10464 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.11.252" port: 36887 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:31:24.964028 10342 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.11.252" port: 36887 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:31:24.964545 10466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:24.964289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:36887 (local address 127.0.0.1:53858)
0808 22:31:24.964386 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:24.964484 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.252:36887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260808 22:31:24.964708 10345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:24.964437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:36887 (local address 127.0.0.1:53872)
0808 22:31:24.964538 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:24.964586 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.252:36887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260808 22:31:24.964784 10380 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.5.11.252:36887: connect: Connection refused (error 111)
I20260808 22:31:24.965478 10345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:24.965215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:36887 (local address 127.0.0.1:53878)
0808 22:31:24.965389 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:24.965428 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.252:36887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
W20260808 22:31:24.965654 10342 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.11.252:36887: Network error: Client connection negotiation failed: client connection to 127.5.11.252:36887: connect: Connection refused (error 111)
I20260808 22:31:24.965731 10466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:24.965402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:36887 (local address 127.0.0.1:53884)
0808 22:31:24.965512 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:24.965559 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.252:36887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260808 22:31:24.965881 10464 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.11.252:36887: Network error: Client connection negotiation failed: client connection to 127.5.11.252:36887: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.5.11.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.252.keytab.
Entry for principal kudu/127.5.11.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.252.keytab.
I20260808 22:31:24.970906 5167 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
/tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.252:36887
--webserver_interface=127.5.11.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.252.keytab
--principal=kudu/127.5.11.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.5.11.212:44039
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.5.11.254:45687,127.5.11.253:40863,127.5.11.252:36887
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260808 22:31:24.996805 10342 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.11.252" port: 36887 } attempt: 1
I20260808 22:31:24.997668 10345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:24.997369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:36887 (local address 127.0.0.1:53886)
0808 22:31:24.997528 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:24.997591 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.252:36887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 22:31:24.998415 10345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:24.998204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:36887 (local address 127.0.0.1:53890)
0808 22:31:24.998314 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:24.998365 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.252:36887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260808 22:31:24.998548 10342 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.11.252:36887: Network error: Client connection negotiation failed: client connection to 127.5.11.252:36887: connect: Connection refused (error 111)
I20260808 22:31:25.015215 10464 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.11.252" port: 36887 } attempt: 1
I20260808 22:31:25.016050 10466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:25.015773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:36887 (local address 127.0.0.1:53896)
0808 22:31:25.015911 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:25.015976 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.252:36887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260808 22:31:25.016765 10466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:25.016554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:36887 (local address 127.0.0.1:53898)
0808 22:31:25.016662 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:25.016713 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.252:36887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260808 22:31:25.016894 10464 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.11.252:36887: Network error: Client connection negotiation failed: client connection to 127.5.11.252:36887: connect: Connection refused (error 111)
I20260808 22:31:25.073719 10342 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.11.252" port: 36887 } attempt: 2
I20260808 22:31:25.074618 10345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:25.074306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:36887 (local address 127.0.0.1:53908)
0808 22:31:25.074466 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:25.074535 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.252:36887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260808 22:31:25.075412 10345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:25.075177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:36887 (local address 127.0.0.1:53918)
0808 22:31:25.075289 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:25.075339 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.252:36887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260808 22:31:25.075532 10342 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.11.252:36887: Network error: Client connection negotiation failed: client connection to 127.5.11.252:36887: connect: Connection refused (error 111)
I20260808 22:31:25.085057 10464 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.11.252" port: 36887 } attempt: 2
I20260808 22:31:25.085767 10466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:25.085517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:36887 (local address 127.0.0.1:53928)
0808 22:31:25.085647 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:25.085699 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.252:36887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 22:31:25.086438 10466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:25.086238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:36887 (local address 127.0.0.1:53938)
0808 22:31:25.086341 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:25.086390 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.5.11.252:36887: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260808 22:31:25.086560 10464 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.5.11.252:36887: Network error: Client connection negotiation failed: client connection to 127.5.11.252:36887: connect: Connection refused (error 111)
W20260808 22:31:25.088002 10473 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260808 22:31:25.088294 10473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260808 22:31:25.088351 10473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260808 22:31:25.088426 10473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260808 22:31:25.092386 10473 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260808 22:31:25.092469 10473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260808 22:31:25.092517 10473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260808 22:31:25.092559 10473 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260808 22:31:25.092607 10473 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260808 22:31:25.098122 10473 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.5.11.212:44039
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.5.11.254:45687,127.5.11.253:40863,127.5.11.252:36887
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/krb5kdc/kudu_127.5.11.252.keytab
--principal=kudu/127.5.11.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.5.11.252:36887
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.5.11.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 08 Aug 2026 21:47:48 UTC on ccycloud.quasar-pjuikp.root.comops.site
build id 4575493
I20260808 22:31:25.099190 10473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260808 22:31:25.100283 10473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260808 22:31:25.108561 10485 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:25.108556 10484 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260808 22:31:25.108558 10488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260808 22:31:25.108805 10473 server_base.cc:1034] running on GCE node
I20260808 22:31:25.109532 10473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260808 22:31:25.110625 10473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260808 22:31:25.111824 10473 hybrid_clock.cc:648] HybridClock initialized: now 1786228285111799 us; error 28 us; skew 500 ppm
Aug 08 22:31:25 dist-test-slave-7f01 krb5kdc[10196](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228285, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260808 22:31:25.114576 10473 init.cc:377] Logged in from keytab as kudu/127.5.11.252@KRBTEST.COM (short username kudu)
I20260808 22:31:25.115489 10473 webserver.cc:460] Webserver started at http://127.5.11.252:43717/ using document root <none> and password file <none>
I20260808 22:31:25.116115 10473 fs_manager.cc:359] Metadata directory not provided
I20260808 22:31:25.116210 10473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260808 22:31:25.116436 10473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260808 22:31:25.118448 10473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-2/data/instance:
uuid: "0d6582e1c0ad49f39c9902bcea3bcba9"
format_stamp: "Formatted at 2026-08-08 22:31:25 on dist-test-slave-7f01"
server_key: "976bbd1fb0d7bf981ed7c7cf50edf1fb"
server_key_iv: "98eeadf0d25f6ff7ac30e8e3d39562f8"
server_key_version: "encryptionkey@0"
I20260808 22:31:25.119014 10473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-2/wal/instance:
uuid: "0d6582e1c0ad49f39c9902bcea3bcba9"
format_stamp: "Formatted at 2026-08-08 22:31:25 on dist-test-slave-7f01"
server_key: "976bbd1fb0d7bf981ed7c7cf50edf1fb"
server_key_iv: "98eeadf0d25f6ff7ac30e8e3d39562f8"
server_key_version: "encryptionkey@0"
I20260808 22:31:25.122714 10473 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260808 22:31:25.125108 10499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260808 22:31:25.126240 10473 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260808 22:31:25.126389 10473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-2/data,/tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-2/wal
uuid: "0d6582e1c0ad49f39c9902bcea3bcba9"
format_stamp: "Formatted at 2026-08-08 22:31:25 on dist-test-slave-7f01"
server_key: "976bbd1fb0d7bf981ed7c7cf50edf1fb"
server_key_iv: "98eeadf0d25f6ff7ac30e8e3d39562f8"
server_key_version: "encryptionkey@0"
I20260808 22:31:25.126502 10473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260808 22:31:25.137859 10473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260808 22:31:25.140731 10473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260808 22:31:25.140937 10473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260808 22:31:25.148077 10473 rpc_server.cc:307] RPC server started. Bound to: 127.5.11.252:36887
I20260808 22:31:25.148370 10589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.5.11.252:36887 every 8 connection(s)
I20260808 22:31:25.149030 10473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-2/data/info.pb
I20260808 22:31:25.149657 5167 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu as pid 10473
I20260808 22:31:25.149824 5167 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskM4Eo1i/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786227991988852-5167-0/minicluster-data/master-2/wal/instance
I20260808 22:31:25.150141 5167 external_mini_cluster.cc:1428] Setting key bd4197359afd95b234fdede57ac7dbd1
I20260808 22:31:25.152356 10591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260808 22:31:25.156958 10591 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.11.254" port: 45687 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 22:31:25 dist-test-slave-7f01 krb5kdc[10196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.252@KRBTEST.COM
Aug 08 22:31:25 dist-test-slave-7f01 krb5kdc[10196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228285, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.252@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:31:25.165328 10597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:25.151509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53940 (local address 127.5.11.252:36887)
0808 22:31:25.151942 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:25.151961 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:25.152002 (+ 41us) server_negotiation.cc:407] Connection header received
0808 22:31:25.152847 (+ 845us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:25.152871 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:25.153282 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:25.153632 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:25.154327 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:25.155911 (+ 1584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:25.156716 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:25.157418 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:25.157565 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:25.160419 (+ 2854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:25.160426 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:25.160437 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:25.160470 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:25.162308 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:25.162775 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:25.162779 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:25.162783 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:25.162943 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:25.163226 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:25.163230 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:25.163232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:25.163827 (+ 595us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:25.164025 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:25.164295 (+ 270us) server_negotiation.cc:299] Negotiation successful
0808 22:31:25.164594 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":115,"threads_started":1}
I20260808 22:31:25.169380 10346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:25.159315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49184 (local address 127.5.11.254:45687)
0808 22:31:25.159490 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:25.159495 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:25.159506 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:31:25.159542 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:25.159544 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:25.159580 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:25.159672 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:25.160432 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:25.161077 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:25.162357 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:25.162748 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:25.162842 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:25.165161 (+ 2319us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:25.165164 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:25.165166 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:25.165188 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:25.167481 (+ 2293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:25.168042 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:25.168045 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:25.168046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:25.168127 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:25.168443 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:25.168447 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:25.168449 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:25.168699 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:25.168810 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:25.169168 (+ 358us) server_negotiation.cc:299] Negotiation successful
0808 22:31:25.169273 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260808 22:31:25.169438 10598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:25.158747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.254:45687 (local address 127.0.0.1:49184)
0808 22:31:25.159203 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:25.159261 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:25.159374 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:25.159725 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:25.159731 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:25.160157 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:25.160278 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:25.160284 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:25.161204 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:25.161211 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:25.162216 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:25.162230 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:25.162872 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:25.162875 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:25.162991 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:25.163447 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:25.163460 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:25.165017 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:25.167589 (+ 2572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:25.167599 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:25.167605 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:25.167942 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:25.168218 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:25.168221 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:25.168223 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:25.168324 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:25.168812 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:25.168817 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:25.168907 (+ 90us) client_negotiation.cc:769] Sending connection context
0808 22:31:25.169054 (+ 147us) client_negotiation.cc:240] Negotiation successful
0808 22:31:25.169234 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":123,"threads_started":1}
I20260808 22:31:25.170673 10591 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.11.253" port: 40863 } has no permanent_uuid. Determining permanent_uuid...
Aug 08 22:31:25 dist-test-slave-7f01 krb5kdc[10196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.254@KRBTEST.COM
I20260808 22:31:25.174666 10342 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.11.252" port: 36887 } attempt: 3
Aug 08 22:31:25 dist-test-slave-7f01 krb5kdc[10196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228285, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.252@KRBTEST.COM for kudu/127.5.11.253@KRBTEST.COM
I20260808 22:31:25.177718 10464 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.5.11.252" port: 36887 } attempt: 3
I20260808 22:31:25.177870 10599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:25.167719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49198 (local address 127.5.11.254:45687)
0808 22:31:25.168108 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:25.168116 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:25.168127 (+ 11us) server_negotiation.cc:407] Connection header received
0808 22:31:25.168157 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:25.168159 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:25.168222 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:25.168311 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:25.169030 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:25.169785 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:25.170672 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:25.170956 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:25.171110 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:25.173641 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:25.173644 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:25.173647 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:25.173678 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:25.175510 (+ 1832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:25.176019 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:25.176023 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:25.176025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:25.176110 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:25.176447 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:25.176453 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:25.176456 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:25.176642 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:25.176852 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:25.177595 (+ 743us) server_negotiation.cc:299] Negotiation successful
0808 22:31:25.177702 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":97,"threads_started":1}
Aug 08 22:31:25 dist-test-slave-7f01 krb5kdc[10196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228284, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.254@KRBTEST.COM for kudu/127.5.11.252@KRBTEST.COM
I20260808 22:31:25.181092 10471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:25.171468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59858 (local address 127.5.11.253:40863)
0808 22:31:25.171580 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:25.171584 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:25.171594 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:31:25.171630 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:25.171632 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:25.171669 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:25.171725 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:25.172302 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:25.172795 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:25.173601 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:25.173835 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:25.173943 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:25.176212 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:25.176216 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:25.176218 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:25.176248 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:25.179145 (+ 2897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:25.179656 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:25.179659 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:25.179661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:25.179770 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:25.180087 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:25.180091 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:25.180093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:25.180297 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:25.180369 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:25.180784 (+ 415us) server_negotiation.cc:299] Negotiation successful
0808 22:31:25.180905 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260808 22:31:25.181488 10598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:25.171185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.253:40863 (local address 127.0.0.1:59858)
0808 22:31:25.171299 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:25.171310 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:25.171394 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:25.171803 (+ 409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:25.171809 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:25.171984 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:25.172118 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:25.172128 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:25.172909 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:25.172916 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:25.173482 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:25.173491 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:25.173945 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:25.173948 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:25.174057 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:25.174407 (+ 350us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:25.174418 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:25.176084 (+ 1666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:25.179275 (+ 3191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:25.179278 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:25.179280 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:25.179540 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:25.179892 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:25.179895 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:25.179896 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:25.179989 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:25.180453 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:25.180457 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:25.180579 (+ 122us) client_negotiation.cc:769] Sending connection context
0808 22:31:25.181206 (+ 627us) client_negotiation.cc:240] Negotiation successful
0808 22:31:25.181330 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260808 22:31:25.182240 10591 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.5.11.252" port: 36887 } has no permanent_uuid. Determining permanent_uuid...
I20260808 22:31:25.184932 10345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:25.175073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:36887 (local address 127.0.0.1:53952)
0808 22:31:25.175574 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:25.175590 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:25.175664 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:25.175968 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:25.175974 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:25.176154 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:25.176252 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:25.176262 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:25.177258 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:25.177261 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:25.178035 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:25.178067 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:25.178973 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:25.178976 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:25.179055 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:25.179477 (+ 422us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:25.179491 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:25.180979 (+ 1488us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:25.183276 (+ 2297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:25.183279 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:25.183280 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:25.183535 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:25.183885 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:25.183889 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:25.183891 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:25.183999 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:25.184456 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:25.184463 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:25.184554 (+ 91us) client_negotiation.cc:769] Sending connection context
0808 22:31:25.184643 (+ 89us) client_negotiation.cc:240] Negotiation successful
0808 22:31:25.184774 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415}
Aug 08 22:31:25 dist-test-slave-7f01 krb5kdc[10196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228284, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.253@KRBTEST.COM for kudu/127.5.11.252@KRBTEST.COM
Aug 08 22:31:25 dist-test-slave-7f01 krb5kdc[10196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228284, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.5.11.253@KRBTEST.COM
I20260808 22:31:25.187496 10597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:25.175114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53952 (local address 127.5.11.252:36887)
0808 22:31:25.175209 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:25.175212 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:25.175650 (+ 438us) server_negotiation.cc:407] Connection header received
0808 22:31:25.175723 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:25.175726 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:25.175792 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:25.176366 (+ 574us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:25.176455 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:25.177116 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:25.178473 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:25.178826 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:25.179264 (+ 438us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:25.181108 (+ 1844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:25.181112 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:25.181114 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:25.181137 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:25.183133 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:25.183634 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:25.183637 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:25.183639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:25.183720 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:25.184114 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:25.184116 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:25.184118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:25.184340 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:25.187073 (+ 2733us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:25.187152 (+ 79us) server_negotiation.cc:299] Negotiation successful
0808 22:31:25.187270 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":37,"spinlock_wait_cycles":18176}
Aug 08 22:31:25 dist-test-slave-7f01 krb5kdc[10196](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786228285, etypes {rep=17 tkt=17 ses=17}, kudu/127.5.11.252@KRBTEST.COM for kudu/127.5.11.252@KRBTEST.COM
I20260808 22:31:25.190810 10466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:25.178111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:36887 (local address 127.0.0.1:53966)
0808 22:31:25.178278 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:25.178295 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:25.178371 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:25.178740 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:25.178743 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:25.178995 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:25.179081 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:25.179088 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:25.181026 (+ 1938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:25.181031 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:25.183077 (+ 2046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:25.183090 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:25.184060 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:25.184067 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:25.184155 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:25.184599 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:25.184615 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:25.186376 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:25.189095 (+ 2719us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:25.189098 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:25.189100 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:25.189332 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:25.189710 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:25.189716 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:25.189721 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:25.189832 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:25.190289 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:25.190292 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:25.190411 (+ 119us) client_negotiation.cc:769] Sending connection context
0808 22:31:25.190512 (+ 101us) client_negotiation.cc:240] Negotiation successful
0808 22:31:25.190658 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":30,"spinlock_wait_cycles":2432}
I20260808 22:31:25.191437 10600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:25.178176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53966 (local address 127.5.11.252:36887)
0808 22:31:25.178534 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:25.178542 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:25.178552 (+ 10us) server_negotiation.cc:407] Connection header received
0808 22:31:25.178579 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:25.178581 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:25.178615 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:25.179190 (+ 575us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:25.179540 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:25.180890 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:25.183532 (+ 2642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:25.183926 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:25.184356 (+ 430us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:25.186514 (+ 2158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:25.186518 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:25.186520 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:25.186544 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:25.188967 (+ 2423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:25.189463 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:25.189467 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:25.189468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:25.189592 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:25.189958 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:25.189962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:25.189965 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:25.190166 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:25.191066 (+ 900us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:25.191155 (+ 89us) server_negotiation.cc:299] Negotiation successful
0808 22:31:25.191267 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"spinlock_wait_cycles":32640,"thread_start_us":152,"threads_started":1}
I20260808 22:31:25.191463 10468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:25.179564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59868 (local address 127.5.11.253:40863)
0808 22:31:25.179671 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:25.179674 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:25.179846 (+ 172us) server_negotiation.cc:407] Connection header received
0808 22:31:25.180035 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:25.180039 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:25.180086 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:25.180136 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:25.180965 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:25.181666 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:25.182788 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:25.183357 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:25.183429 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:25.187035 (+ 3606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:25.187040 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:25.187042 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:25.187067 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:25.188772 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:25.189353 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:25.189360 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:25.189364 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:25.189454 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:25.189789 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:25.189793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:25.189794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:25.190057 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:25.190282 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:25.191177 (+ 895us) server_negotiation.cc:299] Negotiation successful
0808 22:31:25.191291 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":177,"server-negotiator.queue_time_us":46,"spinlock_wait_cycles":3456}
I20260808 22:31:25.191862 10342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42f3c83e63004f3baabecd7b35170fa6: Bootstrap starting.
I20260808 22:31:25.192934 10598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:25.182685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.5.11.252:36887 (local address 127.0.0.1:53968)
0808 22:31:25.183100 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0808 22:31:25.183110 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0808 22:31:25.183164 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0808 22:31:25.184283 (+ 1119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0808 22:31:25.184286 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0808 22:31:25.184461 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0808 22:31:25.184562 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:25.184571 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:25.185624 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:25.185630 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:25.186189 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0808 22:31:25.186206 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0808 22:31:25.186708 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0808 22:31:25.186711 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0808 22:31:25.186801 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:25.187245 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0808 22:31:25.187257 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0808 22:31:25.189328 (+ 2071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0808 22:31:25.191355 (+ 2027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:25.191358 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:25.191360 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:25.191598 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:25.191918 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0808 22:31:25.191922 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0808 22:31:25.191924 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0808 22:31:25.192068 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0808 22:31:25.192497 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0808 22:31:25.192500 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0808 22:31:25.192585 (+ 85us) client_negotiation.cc:769] Sending connection context
0808 22:31:25.192682 (+ 97us) client_negotiation.cc:240] Negotiation successful
0808 22:31:25.192797 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354}
I20260808 22:31:25.193465 10604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:25.182871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53968 (local address 127.5.11.252:36887)
0808 22:31:25.183933 (+ 1062us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:25.183943 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:25.183957 (+ 14us) server_negotiation.cc:407] Connection header received
0808 22:31:25.184010 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:25.184013 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:25.184055 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:25.184142 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:25.184694 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:25.185523 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:25.186306 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:25.186609 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:25.186710 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:25.189452 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:25.189455 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:25.189458 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:25.189492 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:25.191244 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:25.191700 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:25.191703 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:25.191705 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:25.191823 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:25.192167 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:25.192170 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:25.192171 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:25.192358 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:25.192446 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:25.193179 (+ 733us) server_negotiation.cc:299] Negotiation successful
0808 22:31:25.193307 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":990,"thread_start_us":128,"threads_started":1}
I20260808 22:31:25.194201 10464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 394cdb68baf744298ee7068eb138e07c: Bootstrap starting.
I20260808 22:31:25.194717 10342 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42f3c83e63004f3baabecd7b35170fa6: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:25.195168 5167 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260808 22:31:25.195617 10342 log.cc:824] T 00000000000000000000000000000000 P 42f3c83e63004f3baabecd7b35170fa6: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:25.196616 10464 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 394cdb68baf744298ee7068eb138e07c: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:25.196302 10295 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "42f3c83e63004f3baabecd7b35170fa6"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49198
I20260808 22:31:25.197307 10464 log.cc:824] T 00000000000000000000000000000000 P 394cdb68baf744298ee7068eb138e07c: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:25.200008 10342 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42f3c83e63004f3baabecd7b35170fa6: No bootstrap required, opened a new log
I20260808 22:31:25.200074 10591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d6582e1c0ad49f39c9902bcea3bcba9: Bootstrap starting.
I20260808 22:31:25.201002 10464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 394cdb68baf744298ee7068eb138e07c: No bootstrap required, opened a new log
I20260808 22:31:25.202469 10591 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d6582e1c0ad49f39c9902bcea3bcba9: Neither blocks nor log segments found. Creating new log.
I20260808 22:31:25.203006 10342 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42f3c83e63004f3baabecd7b35170fa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42f3c83e63004f3baabecd7b35170fa6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 45687 } } peers { permanent_uuid: "394cdb68baf744298ee7068eb138e07c" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 40863 } } peers { permanent_uuid: "0d6582e1c0ad49f39c9902bcea3bcba9" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 36887 } }
I20260808 22:31:25.203229 10342 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42f3c83e63004f3baabecd7b35170fa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42f3c83e63004f3baabecd7b35170fa6, State: Initialized, Role: FOLLOWER
I20260808 22:31:25.203254 10591 log.cc:824] T 00000000000000000000000000000000 P 0d6582e1c0ad49f39c9902bcea3bcba9: Log is configured to *not* fsync() on all Append() calls
I20260808 22:31:25.203554 10342 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42f3c83e63004f3baabecd7b35170fa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42f3c83e63004f3baabecd7b35170fa6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 45687 } } peers { permanent_uuid: "394cdb68baf744298ee7068eb138e07c" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 40863 } } peers { permanent_uuid: "0d6582e1c0ad49f39c9902bcea3bcba9" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 36887 } }
I20260808 22:31:25.204092 10613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42f3c83e63004f3baabecd7b35170fa6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42f3c83e63004f3baabecd7b35170fa6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 45687 } } peers { permanent_uuid: "394cdb68baf744298ee7068eb138e07c" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 40863 } } peers { permanent_uuid: "0d6582e1c0ad49f39c9902bcea3bcba9" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 36887 } } }
I20260808 22:31:25.204236 10613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42f3c83e63004f3baabecd7b35170fa6 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:31:25.204943 10342 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42f3c83e63004f3baabecd7b35170fa6 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:31:25.204984 10295 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42f3c83e63004f3baabecd7b35170fa6 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260808 22:31:25.205078 10295 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42f3c83e63004f3baabecd7b35170fa6 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:25.205763 10591 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d6582e1c0ad49f39c9902bcea3bcba9: No bootstrap required, opened a new log
I20260808 22:31:25.205932 10464 raft_consensus.cc:348] T 00000000000000000000000000000000 P 394cdb68baf744298ee7068eb138e07c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42f3c83e63004f3baabecd7b35170fa6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 45687 } } peers { permanent_uuid: "394cdb68baf744298ee7068eb138e07c" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 40863 } } peers { permanent_uuid: "0d6582e1c0ad49f39c9902bcea3bcba9" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 36887 } }
I20260808 22:31:25.206179 10464 raft_consensus.cc:729] T 00000000000000000000000000000000 P 394cdb68baf744298ee7068eb138e07c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 394cdb68baf744298ee7068eb138e07c, State: Initialized, Role: FOLLOWER
I20260808 22:31:25.206377 10295 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42f3c83e63004f3baabecd7b35170fa6 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42f3c83e63004f3baabecd7b35170fa6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 45687 } } peers { permanent_uuid: "394cdb68baf744298ee7068eb138e07c" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 40863 } } peers { permanent_uuid: "0d6582e1c0ad49f39c9902bcea3bcba9" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 36887 } }
I20260808 22:31:25.206488 10464 consensus_queue.cc:260] T 00000000000000000000000000000000 P 394cdb68baf744298ee7068eb138e07c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42f3c83e63004f3baabecd7b35170fa6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 45687 } } peers { permanent_uuid: "394cdb68baf744298ee7068eb138e07c" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 40863 } } peers { permanent_uuid: "0d6582e1c0ad49f39c9902bcea3bcba9" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 36887 } }
I20260808 22:31:25.207401 10295 leader_election.cc:288] T 00000000000000000000000000000000 P 42f3c83e63004f3baabecd7b35170fa6 [CANDIDATE]: Term 1 election: Requested vote from peers 394cdb68baf744298ee7068eb138e07c (127.5.11.253:40863), 0d6582e1c0ad49f39c9902bcea3bcba9 (127.5.11.252:36887)
I20260808 22:31:25.207787 10464 sys_catalog.cc:564] T 00000000000000000000000000000000 P 394cdb68baf744298ee7068eb138e07c [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:31:25.207877 10420 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "42f3c83e63004f3baabecd7b35170fa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "394cdb68baf744298ee7068eb138e07c"
I20260808 22:31:25.208128 10420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 394cdb68baf744298ee7068eb138e07c [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:25.209340 10420 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 394cdb68baf744298ee7068eb138e07c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42f3c83e63004f3baabecd7b35170fa6 in term 1.
I20260808 22:31:25.209841 10258 leader_election.cc:302] T 00000000000000000000000000000000 P 42f3c83e63004f3baabecd7b35170fa6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 394cdb68baf744298ee7068eb138e07c, 42f3c83e63004f3baabecd7b35170fa6; no voters:
I20260808 22:31:25.210779 10613 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42f3c83e63004f3baabecd7b35170fa6 [term 1 FOLLOWER]: Leader election won for term 1
I20260808 22:31:25.210995 10613 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42f3c83e63004f3baabecd7b35170fa6 [term 1 LEADER]: Becoming Leader. State: Replica: 42f3c83e63004f3baabecd7b35170fa6, State: Running, Role: LEADER
I20260808 22:31:25.211299 10613 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42f3c83e63004f3baabecd7b35170fa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42f3c83e63004f3baabecd7b35170fa6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 45687 } } peers { permanent_uuid: "394cdb68baf744298ee7068eb138e07c" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 40863 } } peers { permanent_uuid: "0d6582e1c0ad49f39c9902bcea3bcba9" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 36887 } }
I20260808 22:31:25.211977 10591 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d6582e1c0ad49f39c9902bcea3bcba9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42f3c83e63004f3baabecd7b35170fa6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 45687 } } peers { permanent_uuid: "394cdb68baf744298ee7068eb138e07c" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 40863 } } peers { permanent_uuid: "0d6582e1c0ad49f39c9902bcea3bcba9" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 36887 } }
I20260808 22:31:25.212301 10591 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d6582e1c0ad49f39c9902bcea3bcba9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d6582e1c0ad49f39c9902bcea3bcba9, State: Initialized, Role: FOLLOWER
I20260808 22:31:25.212742 10591 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d6582e1c0ad49f39c9902bcea3bcba9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42f3c83e63004f3baabecd7b35170fa6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 45687 } } peers { permanent_uuid: "394cdb68baf744298ee7068eb138e07c" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 40863 } } peers { permanent_uuid: "0d6582e1c0ad49f39c9902bcea3bcba9" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 36887 } }
W20260808 22:31:25.212972 10641 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 394cdb68baf744298ee7068eb138e07c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:31:25.213035 10641 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:31:25.213464 10613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42f3c83e63004f3baabecd7b35170fa6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42f3c83e63004f3baabecd7b35170fa6. Latest consensus state: current_term: 1 leader_uuid: "42f3c83e63004f3baabecd7b35170fa6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42f3c83e63004f3baabecd7b35170fa6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 45687 } } peers { permanent_uuid: "394cdb68baf744298ee7068eb138e07c" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 40863 } } peers { permanent_uuid: "0d6582e1c0ad49f39c9902bcea3bcba9" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 36887 } } }
I20260808 22:31:25.213570 10613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42f3c83e63004f3baabecd7b35170fa6 [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:25.214450 10659 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260808 22:31:25.214705 10548 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "42f3c83e63004f3baabecd7b35170fa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "0d6582e1c0ad49f39c9902bcea3bcba9"
I20260808 22:31:25.214941 10548 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d6582e1c0ad49f39c9902bcea3bcba9 [term 0 FOLLOWER]: Advancing to term 1
I20260808 22:31:25.212177 10357 master_runner.cc:186] Error getting master registration for 127.5.11.254:45687: OK, instance_id { permanent_uuid: "42f3c83e63004f3baabecd7b35170fa6" instance_seqno: 1786228284707676 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 22:31:25.215466 10357 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260808 22:31:25.216289 10548 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0d6582e1c0ad49f39c9902bcea3bcba9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 42f3c83e63004f3baabecd7b35170fa6 in term 1.
I20260808 22:31:25.214651 10591 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d6582e1c0ad49f39c9902bcea3bcba9 [sys.catalog]: configured and running, proceeding with master startup.
I20260808 22:31:25.217453 10659 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260808 22:31:25.219489 10657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d6582e1c0ad49f39c9902bcea3bcba9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42f3c83e63004f3baabecd7b35170fa6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 45687 } } peers { permanent_uuid: "394cdb68baf744298ee7068eb138e07c" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 40863 } } peers { permanent_uuid: "0d6582e1c0ad49f39c9902bcea3bcba9" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 36887 } } }
I20260808 22:31:25.219623 10657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d6582e1c0ad49f39c9902bcea3bcba9 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:31:25.219898 10619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 394cdb68baf744298ee7068eb138e07c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42f3c83e63004f3baabecd7b35170fa6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 45687 } } peers { permanent_uuid: "394cdb68baf744298ee7068eb138e07c" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 40863 } } peers { permanent_uuid: "0d6582e1c0ad49f39c9902bcea3bcba9" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 36887 } } }
I20260808 22:31:25.220017 10619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 394cdb68baf744298ee7068eb138e07c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:31:25.219117 10222 master_runner.cc:186] Error getting master registration for 127.5.11.252:36887: OK, instance_id { permanent_uuid: "0d6582e1c0ad49f39c9902bcea3bcba9" instance_seqno: 1786228285146103 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260808 22:31:25.222565 10222 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260808 22:31:25.231374 10684 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0d6582e1c0ad49f39c9902bcea3bcba9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260808 22:31:25.231460 10684 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260808 22:31:25.232152 10468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:25.216146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59870 (local address 127.5.11.253:40863)
0808 22:31:25.216256 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:25.216261 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:25.218205 (+ 1944us) server_negotiation.cc:407] Connection header received
0808 22:31:25.218395 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:25.218398 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:25.218443 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:25.218503 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:25.221744 (+ 3241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:25.222500 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:25.225776 (+ 3276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:25.226092 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:25.226172 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:25.228274 (+ 2102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:25.228278 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:25.228281 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:25.228307 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:25.229983 (+ 1676us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:25.230616 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:25.230620 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:25.230622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:25.230714 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:25.231054 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:25.231057 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:25.231059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:25.231299 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:25.231418 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:25.231856 (+ 438us) server_negotiation.cc:299] Negotiation successful
0808 22:31:25.231985 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260808 22:31:25.235354 10599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:25.216979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49212 (local address 127.5.11.254:45687)
0808 22:31:25.217101 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:25.217105 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:25.219796 (+ 2691us) server_negotiation.cc:407] Connection header received
0808 22:31:25.219962 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:25.219966 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:25.220010 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:25.220058 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:25.223920 (+ 3862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:25.224643 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:25.226712 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:25.227099 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:25.227177 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:25.229556 (+ 2379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:25.229560 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:25.229563 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:25.229589 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:25.232072 (+ 2483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:25.233857 (+ 1785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:25.233861 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:25.233862 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:25.233964 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:25.234355 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:25.234359 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:25.234360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:25.234594 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:25.234734 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:25.235082 (+ 348us) server_negotiation.cc:299] Negotiation successful
0808 22:31:25.235204 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260808 22:31:25.241828 10600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:25.217804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53972 (local address 127.5.11.252:36887)
0808 22:31:25.217927 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:25.217931 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:25.232618 (+ 14687us) server_negotiation.cc:407] Connection header received
0808 22:31:25.232803 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:25.232807 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:25.232869 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:25.232924 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:25.233688 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:25.234487 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:25.235605 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:25.236007 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:25.236085 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:25.238087 (+ 2002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0808 22:31:25.238092 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0808 22:31:25.238095 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0808 22:31:25.238122 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0808 22:31:25.240024 (+ 1902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:25.240527 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:25.240531 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:25.240533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:25.240625 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0808 22:31:25.240926 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0808 22:31:25.240929 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0808 22:31:25.240932 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0808 22:31:25.241152 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0808 22:31:25.241284 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:25.241539 (+ 255us) server_negotiation.cc:299] Negotiation successful
0808 22:31:25.241667 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260808 22:31:25.249274 10548 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0d6582e1c0ad49f39c9902bcea3bcba9 [term 1 FOLLOWER]: Refusing update from remote peer 42f3c83e63004f3baabecd7b35170fa6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:31:25.249965 10613 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 42f3c83e63004f3baabecd7b35170fa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d6582e1c0ad49f39c9902bcea3bcba9" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 36887 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:31:25.253973 10657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d6582e1c0ad49f39c9902bcea3bcba9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42f3c83e63004f3baabecd7b35170fa6. Latest consensus state: current_term: 1 leader_uuid: "42f3c83e63004f3baabecd7b35170fa6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42f3c83e63004f3baabecd7b35170fa6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 45687 } } peers { permanent_uuid: "394cdb68baf744298ee7068eb138e07c" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 40863 } } peers { permanent_uuid: "0d6582e1c0ad49f39c9902bcea3bcba9" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 36887 } } }
I20260808 22:31:25.254084 10657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d6582e1c0ad49f39c9902bcea3bcba9 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:31:25.257609 10420 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 394cdb68baf744298ee7068eb138e07c [term 1 FOLLOWER]: Refusing update from remote peer 42f3c83e63004f3baabecd7b35170fa6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260808 22:31:25.258176 10647 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 42f3c83e63004f3baabecd7b35170fa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "394cdb68baf744298ee7068eb138e07c" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 40863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260808 22:31:25.261592 10613 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42f3c83e63004f3baabecd7b35170fa6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "42f3c83e63004f3baabecd7b35170fa6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42f3c83e63004f3baabecd7b35170fa6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 45687 } } peers { permanent_uuid: "394cdb68baf744298ee7068eb138e07c" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 40863 } } peers { permanent_uuid: "0d6582e1c0ad49f39c9902bcea3bcba9" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 36887 } } }
I20260808 22:31:25.261703 10613 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42f3c83e63004f3baabecd7b35170fa6 [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:25.262018 10619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 394cdb68baf744298ee7068eb138e07c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42f3c83e63004f3baabecd7b35170fa6. Latest consensus state: current_term: 1 leader_uuid: "42f3c83e63004f3baabecd7b35170fa6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42f3c83e63004f3baabecd7b35170fa6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 45687 } } peers { permanent_uuid: "394cdb68baf744298ee7068eb138e07c" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 40863 } } peers { permanent_uuid: "0d6582e1c0ad49f39c9902bcea3bcba9" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 36887 } } }
I20260808 22:31:25.262135 10619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 394cdb68baf744298ee7068eb138e07c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:31:25.262801 10657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d6582e1c0ad49f39c9902bcea3bcba9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "42f3c83e63004f3baabecd7b35170fa6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42f3c83e63004f3baabecd7b35170fa6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 45687 } } peers { permanent_uuid: "394cdb68baf744298ee7068eb138e07c" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 40863 } } peers { permanent_uuid: "0d6582e1c0ad49f39c9902bcea3bcba9" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 36887 } } }
I20260808 22:31:25.262898 10657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d6582e1c0ad49f39c9902bcea3bcba9 [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:31:25.265373 10688 mvcc.cc:205] Tried to move back new op lower bound from 7316391056317898752 to 7316391056230703104. Current Snapshot: MvccSnapshot[applied={T|T < 7316391056317898752}]
I20260808 22:31:25.265812 10689 mvcc.cc:205] Tried to move back new op lower bound from 7316391056317898752 to 7316391056230703104. Current Snapshot: MvccSnapshot[applied={T|T < 7316391056317898752}]
I20260808 22:31:25.269214 10659 catalog_manager.cc:1347] Generated new cluster ID: a0b3de6a3f32459d89f4fbec4a3ae929
I20260808 22:31:25.269284 10659 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260808 22:31:25.270953 10647 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42f3c83e63004f3baabecd7b35170fa6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "42f3c83e63004f3baabecd7b35170fa6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42f3c83e63004f3baabecd7b35170fa6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 45687 } } peers { permanent_uuid: "394cdb68baf744298ee7068eb138e07c" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 40863 } } peers { permanent_uuid: "0d6582e1c0ad49f39c9902bcea3bcba9" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 36887 } } }
I20260808 22:31:25.271081 10647 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42f3c83e63004f3baabecd7b35170fa6 [sys.catalog]: This master's current role is: LEADER
I20260808 22:31:25.272029 10619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 394cdb68baf744298ee7068eb138e07c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "42f3c83e63004f3baabecd7b35170fa6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "42f3c83e63004f3baabecd7b35170fa6" member_type: VOTER last_known_addr { host: "127.5.11.254" port: 45687 } } peers { permanent_uuid: "394cdb68baf744298ee7068eb138e07c" member_type: VOTER last_known_addr { host: "127.5.11.253" port: 40863 } } peers { permanent_uuid: "0d6582e1c0ad49f39c9902bcea3bcba9" member_type: VOTER last_known_addr { host: "127.5.11.252" port: 36887 } } }
I20260808 22:31:25.272130 10619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 394cdb68baf744298ee7068eb138e07c [sys.catalog]: This master's current role is: FOLLOWER
I20260808 22:31:25.274367 10694 mvcc.cc:205] Tried to move back new op lower bound from 7316391056317898752 to 7316391056230703104. Current Snapshot: MvccSnapshot[applied={T|T < 7316391056317898752}]
I20260808 22:31:25.286119 10659 catalog_manager.cc:1370] Generated new certificate authority record
I20260808 22:31:25.287441 10659 catalog_manager.cc:1504] Loading token signing keys...
I20260808 22:31:25.299095 10659 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42f3c83e63004f3baabecd7b35170fa6: Generated new TSK 0
I20260808 22:31:25.300320 10659 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260808 22:31:25.340696 10468 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:31:25.338036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59872 (local address 127.5.11.253:40863)
0808 22:31:25.338173 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:25.338179 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:25.339541 (+ 1362us) server_negotiation.cc:407] Connection header received
0808 22:31:25.339793 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:25.339797 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:25.339859 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:25.339914 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:25.340557 (+ 643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59872: BlockingRecv error: recv got EOF from 127.0.0.1:59872 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260808 22:31:25.342147 10599 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:31:25.338868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49218 (local address 127.5.11.254:45687)
0808 22:31:25.338987 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:25.338991 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:25.341189 (+ 2198us) server_negotiation.cc:407] Connection header received
0808 22:31:25.341427 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:25.341431 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:25.341481 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:25.341531 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:25.342033 (+ 502us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49218: BlockingRecv error: recv got EOF from 127.0.0.1:49218 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260808 22:31:25.344103 10600 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:31:25.342744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53978 (local address 127.5.11.252:36887)
0808 22:31:25.342869 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:25.342873 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:25.343204 (+ 331us) server_negotiation.cc:407] Connection header received
0808 22:31:25.343354 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:25.343357 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:25.343404 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:25.343454 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:25.343996 (+ 542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53978: BlockingRecv error: recv got EOF from 127.0.0.1:53978 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260808 22:31:25.348281 10468 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:31:25.346735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59874 (local address 127.5.11.253:40863)
0808 22:31:25.346869 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:25.346873 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:25.347340 (+ 467us) server_negotiation.cc:407] Connection header received
0808 22:31:25.347560 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:25.347564 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:25.347638 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:25.347686 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:25.348189 (+ 503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59874: BlockingRecv error: recv got EOF from 127.0.0.1:59874 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260808 22:31:25.352952 10468 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:31:25.351521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59888 (local address 127.5.11.253:40863)
0808 22:31:25.351614 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:25.351618 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:25.352143 (+ 525us) server_negotiation.cc:407] Connection header received
0808 22:31:25.352310 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:25.352313 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:25.352358 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:25.352407 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:25.352865 (+ 458us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59888: BlockingRecv error: recv got EOF from 127.0.0.1:59888 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
W20260808 22:31:25.358125 10468 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:31:25.356747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59896 (local address 127.5.11.253:40863)
0808 22:31:25.356836 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:25.356841 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:25.357330 (+ 489us) server_negotiation.cc:407] Connection header received
0808 22:31:25.357499 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:25.357502 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:25.357544 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:25.357590 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:25.358041 (+ 451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59896: BlockingRecv error: recv got EOF from 127.0.0.1:59896 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
W20260808 22:31:25.365444 10468 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:31:25.363976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59902 (local address 127.5.11.253:40863)
0808 22:31:25.364092 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:25.364096 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:25.364597 (+ 501us) server_negotiation.cc:407] Connection header received
0808 22:31:25.364764 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:25.364767 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:25.364812 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:25.364871 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:25.365354 (+ 483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59902: BlockingRecv error: recv got EOF from 127.0.0.1:59902 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260808 22:31:25.377067 10468 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:31:25.375392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59912 (local address 127.5.11.253:40863)
0808 22:31:25.375526 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:25.375533 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:25.376130 (+ 597us) server_negotiation.cc:407] Connection header received
0808 22:31:25.376319 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:25.376322 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:25.376373 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:25.376422 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:25.376961 (+ 539us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59912: BlockingRecv error: recv got EOF from 127.0.0.1:59912 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260808 22:31:25.397467 10468 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:31:25.395495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59922 (local address 127.5.11.253:40863)
0808 22:31:25.395629 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:25.395634 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:25.396363 (+ 729us) server_negotiation.cc:407] Connection header received
0808 22:31:25.396586 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:25.396590 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:25.396643 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:25.396697 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:25.397336 (+ 639us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59922: BlockingRecv error: recv got EOF from 127.0.0.1:59922 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260808 22:31:25.433732 10468 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:31:25.432055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59932 (local address 127.5.11.253:40863)
0808 22:31:25.432185 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:25.432190 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:25.432484 (+ 294us) server_negotiation.cc:407] Connection header received
0808 22:31:25.432744 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:25.432750 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:25.432823 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:25.432900 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:25.433601 (+ 701us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59932: BlockingRecv error: recv got EOF from 127.0.0.1:59932 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260808 22:31:25.501011 10468 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:31:25.499890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59934 (local address 127.5.11.253:40863)
0808 22:31:25.500046 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:25.500051 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:25.500178 (+ 127us) server_negotiation.cc:407] Connection header received
0808 22:31:25.500308 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:25.500310 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:25.500362 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:25.500446 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:25.500856 (+ 410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59934: BlockingRecv error: recv got EOF from 127.0.0.1:59934 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260808 22:31:25.632589 10468 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:31:25.631435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59950 (local address 127.5.11.253:40863)
0808 22:31:25.631584 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:25.631588 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:25.631778 (+ 190us) server_negotiation.cc:407] Connection header received
0808 22:31:25.631913 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:25.631916 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:25.631967 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:25.632055 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:25.632480 (+ 425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59950: BlockingRecv error: recv got EOF from 127.0.0.1:59950 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260808 22:31:25.891990 10468 negotiation.cc:337] Failed RPC negotiation. Trace:
0808 22:31:25.890932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59958 (local address 127.5.11.253:40863)
0808 22:31:25.891066 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:25.891071 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:25.891145 (+ 74us) server_negotiation.cc:407] Connection header received
0808 22:31:25.891253 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:25.891255 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:25.891303 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:25.891371 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0808 22:31:25.891870 (+ 499us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59958: BlockingRecv error: recv got EOF from 127.0.0.1:59958 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260808 22:31:26.214428 10641 catalog_manager.cc:1259] Loaded cluster ID: a0b3de6a3f32459d89f4fbec4a3ae929
I20260808 22:31:26.214517 10641 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 394cdb68baf744298ee7068eb138e07c: loading cluster ID for follower catalog manager: success
I20260808 22:31:26.216203 10641 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 394cdb68baf744298ee7068eb138e07c: acquiring CA information for follower catalog manager: success
I20260808 22:31:26.217442 10641 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 394cdb68baf744298ee7068eb138e07c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 22:31:26.232618 10684 catalog_manager.cc:1259] Loaded cluster ID: a0b3de6a3f32459d89f4fbec4a3ae929
I20260808 22:31:26.232685 10684 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0d6582e1c0ad49f39c9902bcea3bcba9: loading cluster ID for follower catalog manager: success
I20260808 22:31:26.234256 10684 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0d6582e1c0ad49f39c9902bcea3bcba9: acquiring CA information for follower catalog manager: success
I20260808 22:31:26.235349 10684 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0d6582e1c0ad49f39c9902bcea3bcba9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260808 22:31:26.411018 10814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:26.406191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59960 (local address 127.5.11.253:40863)
0808 22:31:26.406476 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:26.406480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:26.406595 (+ 115us) server_negotiation.cc:407] Connection header received
0808 22:31:26.406794 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:26.406796 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:26.406952 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:26.407086 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:31:26.407405 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:26.408020 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:26.409270 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:26.409674 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:26.409772 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:26.410135 (+ 363us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:31:26.410595 (+ 460us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:31:26.410691 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:26.410765 (+ 74us) server_negotiation.cc:299] Negotiation successful
0808 22:31:26.410812 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":79,"threads_started":1}
I20260808 22:31:26.417915 10824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:26.414066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53984 (local address 127.5.11.252:36887)
0808 22:31:26.414305 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:26.414308 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:26.414365 (+ 57us) server_negotiation.cc:407] Connection header received
0808 22:31:26.414508 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:26.414510 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:26.414623 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:26.414718 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:31:26.414901 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:26.415381 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:26.416277 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:26.416506 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:26.416770 (+ 264us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:26.417036 (+ 266us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:31:26.417440 (+ 404us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:31:26.417537 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:26.417661 (+ 124us) server_negotiation.cc:299] Negotiation successful
0808 22:31:26.417781 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":63,"threads_started":1}
I20260808 22:31:26.424964 10814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:26.420907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59976 (local address 127.5.11.253:40863)
0808 22:31:26.421033 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:26.421038 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:26.421395 (+ 357us) server_negotiation.cc:407] Connection header received
0808 22:31:26.421451 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:26.421454 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:26.421507 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:26.421590 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:31:26.421936 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:26.422673 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:26.423881 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:26.424124 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:26.424232 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:26.424506 (+ 274us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:31:26.424569 (+ 63us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:31:26.424627 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:26.424776 (+ 149us) server_negotiation.cc:299] Negotiation successful
0808 22:31:26.424830 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260808 22:31:26.425671 10824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:26.420982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54000 (local address 127.5.11.252:36887)
0808 22:31:26.421130 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:26.421137 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:26.421607 (+ 470us) server_negotiation.cc:407] Connection header received
0808 22:31:26.421753 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:26.421757 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:26.421808 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:26.421886 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:31:26.422119 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:26.422714 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:26.423919 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:26.424248 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:26.424355 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:26.425188 (+ 833us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:31:26.425261 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:31:26.425324 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:26.425494 (+ 170us) server_negotiation.cc:299] Negotiation successful
0808 22:31:26.425546 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260808 22:31:26.426057 10834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0808 22:31:26.420855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49224 (local address 127.5.11.254:45687)
0808 22:31:26.421379 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0808 22:31:26.421385 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0808 22:31:26.421734 (+ 349us) server_negotiation.cc:407] Connection header received
0808 22:31:26.421831 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0808 22:31:26.421837 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0808 22:31:26.422001 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0808 22:31:26.422106 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0808 22:31:26.422471 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:26.423217 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:26.424361 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0808 22:31:26.424693 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0808 22:31:26.424791 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0808 22:31:26.425144 (+ 353us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0808 22:31:26.425468 (+ 324us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0808 22:31:26.425548 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0808 22:31:26.425829 (+ 281us) server_negotiation.cc:299] Negotiation successful
0808 22:31:26.425900 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":68,"threads_started":1}
I20260808 22:31:26.432399 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 10222
I20260808 22:31:26.449154 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 10357
I20260808 22:31:26.460903 5167 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskM4Eo1i/build/debug/bin/kudu with pid 10473
2026-08-08T22:31:26Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3318 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6701 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (294475 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST